tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.CaVIghnlBC', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 8d4d7268-903c-4e87-aa46-8e462c808565 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 (finished: start all VMs, in 71.48 seconds) gitlab # gitlab: waiting for unit setup-gitlab.service gitlab # Allocating group tables: 0/8 done gitlab: waiting for the VM to finish booting gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/abixm8krzs1bg4nv97nd774bz7iq4cfa-nixos-system-gitlab-test/init regInfo=/nix/store/dayfsqgy8aay79c9jc2q2c96p5qrccdd-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/dayfsqgy8aay79c9jc2q2c96p5qrccdd-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f00000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f10000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000030] arm-pv: using stolen time PV gitlab # [ 0.000369] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000561] Console: colour dummy device 80x25 gitlab # [ 0.000568] printk: legacy console [tty0] enabled gitlab # [ 0.000771] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000777] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000991] landlock: Up and running. gitlab # [ 0.000994] Yama: becoming mindful. gitlab # [ 0.001383] LSM support for eBPF active gitlab # [ 0.001583] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001704] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002889] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003615] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003619] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003771] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004861] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004954] EFI services will not be available. gitlab # [ 0.005123] smp: Bringing up secondary CPUs ... gitlab # [ 0.005802] Detected PIPT I-cache on CPU1 gitlab # [ 0.005903] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.006035] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.006162] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.007282] Detected PIPT I-cache on CPU2 gitlab # [ 0.007391] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007523] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.007651] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008755] Detected PIPT I-cache on CPU3 gitlab # [ 0.008870] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.009002] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.009130] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009649] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009677] SMP: Total of 4 processors activated. gitlab # [ 0.009679] CPU: All CPU(s) started at EL1 gitlab # [ 0.009688] CPU features: detected: Branch Target Identification gitlab # [ 0.009692] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009694] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009698] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009701] CPU features: detected: Common not Private translations gitlab # [ 0.009704] CPU features: detected: CRC32 instructions gitlab # [ 0.009706] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009709] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009712] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009715] CPU features: detected: E0PD gitlab # [ 0.009717] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009719] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009722] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009725] CPU features: detected: Fine Grained Traps gitlab # [ 0.009728] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009731] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009734] CPU features: detected: LSE atomic instructions gitlab # [ 0.009736] CPU features: detected: Privileged Access Never gitlab # [ 0.009739] CPU features: detected: PMUv3 gitlab # [ 0.009741] CPU features: detected: RAS Extension Support gitlab # [ 0.009743] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009746] CPU features: detected: Random Number Generator gitlab # [ 0.009748] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009750] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009753] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009756] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.009889] alternatives: applying system-wide alternatives gitlab # [ 0.012844] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.013083] Memory: 5963808K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.014619] devtmpfs: initialized gitlab # [ 0.018863] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.018935] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.019132] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.019137] 0 pages in range for non-PLT usage gitlab # [ 0.019138] 508288 pages in range for PLT usage gitlab # [ 0.019279] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.020058] DMI not present or invalid. gitlab # [ 0.023369] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.026321] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.026609] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.028647] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.028670] audit: initializing netlink subsys (disabled) gitlab # [ 0.028992] audit: type=2000 audit(0.028:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.029429] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.029432] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.029436] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.029439] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.029442] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.029532] cpuidle: using governor ladder gitlab # [ 0.029557] cpuidle: using governor menu gitlab # [ 0.029794] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.030001] ASID allocator initialised with 65536 entries gitlab # [ 0.031109] Serial: AMBA PL011 UART driver gitlab # [ 0.036315] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.036641] printk: console [ttyAMA0] enabled gitlab # [ 0.046905] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.046912] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.046916] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.046918] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.046921] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.046924] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.046926] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.046929] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.061044] fbcon: Taking over console gitlab # [ 0.061077] ACPI: Interpreter disabled. gitlab # [ 0.063912] iommu: Default domain type: Translated gitlab # [ 0.063920] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.074268] SCSI subsystem initialized gitlab # [ 0.075082] usbcore: registered new interface driver usbfs gitlab # [ 0.075258] usbcore: registered new interface driver hub gitlab # [ 0.075334] usbcore: registered new device driver usb gitlab # [ 0.076132] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.076143] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.076166] PTP clock support registered gitlab # [ 0.076305] EDAC MC: Ver: 3.0.0 gitlab # [ 0.076875] scmi_core: SCMI protocol bus registered gitlab # [ 0.077771] FPGA manager framework gitlab # [ 0.080142] vgaarb: loaded gitlab # [ 0.081162] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.081747] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.081777] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.083669] netfs: FS-Cache loaded gitlab # [ 0.083834] pnp: PnP ACPI: disabled gitlab # [ 0.096582] NET: Registered PF_INET protocol family gitlab # [ 0.097955] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.134507] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.134565] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.134616] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.134901] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.135555] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.135665] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.135720] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.135845] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.136048] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.136066] NET: Registered PF_XDP protocol family gitlab # [ 0.136080] PCI: CLS 0 bytes, default 64 gitlab # [ 0.136418] Trying to unpack rootfs image as initramfs... gitlab # [ 0.149254] kvm [1]: HYP mode not available gitlab # [ 0.201863] Initialise system trusted keyrings gitlab # [ 0.202309] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.205441] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.209224] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.223869] Key type asymmetric registered gitlab # [ 0.223875] Asymmetric key parser 'x509' registered gitlab # [ 0.223961] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.224340] io scheduler mq-deadline registered gitlab # [ 0.224360] io scheduler kyber registered gitlab # [ 0.238095] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.242066] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.243475] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.243513] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.243563] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.243583] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.243674] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.243726] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.243997] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.244036] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.244070] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.244082] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.244091] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.244351] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.245862] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.246532] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.246568] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.246661] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.246697] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.248267] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.248913] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.248959] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.249051] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.250618] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.251276] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.251313] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.251403] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.252967] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.253622] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.253657] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.253736] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.255333] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.255991] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.256026] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.256108] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.257683] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.258288] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.258322] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.258400] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.260032] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.260647] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.260681] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.260762] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.260798] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.262502] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.263149] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.263239] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.264764] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.265419] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.265500] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.266903] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.267559] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.268369] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.269091] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.269125] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.269236] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.270858] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.271476] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.271510] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.271601] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.273527] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.273546] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.273557] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.273700] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.273842] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.273973] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.274125] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.274265] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.274397] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.274541] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.274685] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.274839] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.275005] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.275155] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.275209] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.275263] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.275315] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.275368] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.275433] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.275499] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.275551] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.275605] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.275660] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.275715] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.275771] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.275823] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.275874] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.275926] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.275977] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.276029] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.276080] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.276132] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.276184] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.276237] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.276300] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.276317] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.276326] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.280494] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.313122] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.317898] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.322837] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.329289] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.334236] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.340151] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.345160] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.350120] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.356866] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.361201] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.366848] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.379020] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.382384] msm_serial: driver initialized gitlab # [ 0.382683] SuperH (H)SCI(F) driver initialized gitlab # [ 0.382792] STM32 USART driver initialized gitlab # [ 0.416085] loop: module loaded gitlab # [ 0.416413] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.419762] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.422909] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.425821] virtio_blk virtio5: [vdb] 12285184 512-byte logical blocks (6.29 GB/5.86 GiB) gitlab # [ 0.429049] megasas: 07.734.00.00-rc1 gitlab # [ 0.430315] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.433378] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.433401] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.435971] Using buffer write method gitlab # [ 0.436014] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.438946] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.438969] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.442747] Using buffer write method gitlab # [ 0.442770] Concatenating MTD devices: gitlab # [ 0.442774] (0): "0.flash" gitlab # [ 0.442778] (1): "0.flash" gitlab # [ 0.442781] into device "0.flash" gitlab # [ 0.590956] Freeing initrd memory: 27184K gitlab # [ 0.608482] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.616953] thunder_xcv, ver 1.0 gitlab # [ 0.617062] thunder_bgx, ver 1.0 gitlab # [ 0.617152] nicpf, ver 1.0 gitlab # [ 0.618890] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.618902] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.618980] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.618996] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.619174] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.619187] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.619296] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.619306] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.619768] sky2: driver version 1.30 gitlab # [ 0.624636] usbcore: registered new interface driver usb-storage gitlab # [ 0.624925] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.624962] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.625002] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.625045] usbserial: USB Serial support registered for generic gitlab # [ 0.625808] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.626984] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.629694] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.629751] rtc-pl031 9010000.pl031: setting system clock to 2026-09-13T10:45:25 UTC (1789296325) gitlab # [ 0.630634] i2c_dev: i2c /dev entries driver gitlab # [ 0.637278] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.637988] hub 1-0:1.0: USB hub found gitlab # [ 0.638024] hub 1-0:1.0: 6 ports detected gitlab # [ 0.638938] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.638951] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.639810] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.640950] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.645835] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.646747] usbcore: registered new interface driver usbhid gitlab # [ 0.646758] usbhid: USB HID core driver gitlab # [ 0.680029] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.685038] drop_monitor: Initializing network drop monitor service gitlab # [ 0.685543] NET: Registered PF_INET6 protocol family gitlab # [ 0.688145] Segment Routing with IPv6 gitlab # [ 0.688185] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.688390] NET: Registered PF_PACKET protocol family gitlab # [ 0.688711] 9pnet: Installing 9P2000 support gitlab # [ 0.693393] Key type dns_resolver registered gitlab # [ 0.711320] registered taskstats version 1 gitlab # [ 0.711724] Loading compiled-in X.509 certificates gitlab # [ 0.731683] Demotion targets for Node 0: null gitlab # [ 0.731961] Key type .fscrypt registered gitlab # [ 0.731970] Key type fscrypt-provisioning registered gitlab # [ 0.732155] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.732189] ima: Allocated hash algorithm: sha1 gitlab # [ 0.732230] ima: No architecture policies found gitlab # [ 0.733595] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.772500] clk: Disabling unused clocks gitlab # [ 0.772547] PM: genpd: Disabling unused power domains gitlab # [ 0.781642] Freeing unused kernel memory: 4736K gitlab # [ 0.781978] Run /init as init process gitlab # [ 0.824212] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.889243] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.043880] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.130033] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.159280] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.159422] systemd[1]: Detected virtualization qemu. gitlab # [ 1.159619] systemd[1]: Detected architecture arm64. gitlab # [ 1.159642] systemd[1]: Running in initrd. gitlab # [ 1.161616] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.162216] systemd[1]: Hostname set to . gitlab # [ 1.249343] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.405762] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.406425] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.451141] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.577437] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.593334] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.593754] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.593848] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.593864] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.593891] systemd[1]: Reached target Path Units. gitlab # [ 1.593908] systemd[1]: Reached target Slice Units. gitlab # [ 1.593924] systemd[1]: Reached target Swaps. gitlab # [ 1.593942] systemd[1]: Reached target Timer Units. gitlab # [ 1.594192] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.594458] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.594713] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.594976] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.595177] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.595235] systemd[1]: Reached target Socket Units. gitlab # [ 1.598189] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.601110] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.601297] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.604132] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.636000] systemd[1]: Starting Journal Service... gitlab # [ 1.641368] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.641504] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.645822] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.648056] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.649594] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.650075] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.650588] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.652944] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.673151] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.675119] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.693708] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.693934] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.693974] systemd[1]: Reached target Local File Systems. gitlab # [ 1.701676] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.709183] systemd-journald[94]: Collecting audit messages is disabled. gitlab # [ 1.710282] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.710594] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.717018] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.717246] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.717250] [drm] features: -context_init gitlab # [ 1.718212] [drm] number of scanouts: 1 gitlab # [ 1.718223] [drm] number of cap sets: 0 gitlab # [ 1.718938] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.718945] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.729711] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.736890] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.756849] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.765609] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.793617] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.806697] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.827401] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.828741] systemd[1]: Started Journal Service. gitlab # [ 1.823726] systemd-modules-load[95]: Using 4 probe threads gitlab # [ 1.826879] systemd-modules-load[95]: Module 'virtio_balloon' is built in gitlab # [ 1.828025] systemd-modules-load[95]: Module 'virtio_console' is built in gitlab # [ 1.829244] systemd-modules-load[95]: Module 'virtio_rng' is built in gitlab # [ 1.830481] systemd-modules-load[95]: Inserted module 'dm_mod' gitlab # [ 1.831535] systemd-modules-load[95]: Inserted module 'virtio_gpu' gitlab # [ 1.833069] systemd-udevd[104]: Using default interface naming scheme 'v261'. gitlab # [ 1.834826] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.864122] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.883535] systemd-vconsole-setup[120]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.886154] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.550171] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 3.046412] EXT4-fs (vda): mounted filesystem 8d4d7268-903c-4e87-aa46-8e462c808565 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.553169] systemd[1]: Reached target System Initialization. gitlab # [ 3.177387] systemd[1]: Reached target Basic System. gitlab # [ 3.178644] (udev-worker)[113]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 3.181122] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 3.182149] (udev-worker)[113]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 3.183686] (udev-worker)[143]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 3.184910] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 3.185718] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 3.186569] systemd[1]: Reached target Initrd Root Device. gitlab # [ 3.187329] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 3.188405] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 3.189411] systemd[1]: Mounting /sysroot... gitlab # [ 3.190061] systemd[1]: Mounted /sysroot. gitlab # [ 3.190677] systemd[1]: Reached target Initrd Root File System. gitlab # [ 3.191519] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 3.192786] systemd-sysroot-fstab-check[157]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 3.194203] systemd[1]: Reload requested from client PID 157 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 3.195720] systemd[1]: Reloading... gitlab # [ 3.196336] systemd[1]: Reloading finished in 104 ms. gitlab # [ 3.244743] systemd-sysroot-fstab-check[157]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.246423] systemd-sysroot-fstab-check[157]: Requesting swap.target/start/replace... gitlab # [ 3.249283] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.250955] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.252610] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.253985] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.267180] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.269150] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.457763] (udev-worker)[112]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 3.460888] (udev-worker)[112]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 3.470964] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.472454] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.473213] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 3.473990] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.509415] systemd-vconsole-setup[181]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 3.511576] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.648543] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 3.652196] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 3.658491] systemd[1]: Mounting /sysroot/run... gitlab # [ 3.672909] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 3.680583] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 3.700801] systemd[1]: Mounted /sysroot/run. gitlab # [ 3.703119] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 3.713747] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.712983] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.720903] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.723429] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 3.727946] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.739634] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.742861] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.647687] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 4.710187] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 4.712632] systemd[1]: Reached target Initrd File Systems. gitlab # [ 4.715082] systemd[1]: Starting Find NixOS closure... gitlab # [ 4.718323] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 4.760321] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 4.763940] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 4.785169] systemd[1]: Finished Find NixOS closure. gitlab # [ 4.787484] systemd[1]: Reached target Initrd Default Target. gitlab # [ 4.791141] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 4.839603] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 4.842373] systemd[1]: Stopped target Basic System. gitlab # [ 4.844648] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 4.847039] systemd[1]: Stopped target Path Units. gitlab # [ 4.849238] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 4.852463] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 4.855740] systemd[1]: Stopped target Slice Units. gitlab # [ 4.858697] systemd[1]: Stopped target Socket Units. gitlab # [ 4.865800] systemd[1]: Stopped target System Initialization. gitlab # [ 4.870614] systemd[1]: Stopped target Swaps. gitlab # [ 4.875986] systemd[1]: Stopped target Timer Units. gitlab # [ 4.880412] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 4.882978] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 4.885658] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 4.888580] systemd[1]: Stopped Find NixOS closure. gitlab # [ 4.890614] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 4.893217] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 4.895506] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 4.898206] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 4.900372] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 4.903283] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 4.905330] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 4.908117] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 4.910833] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 4.913440] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 4.915431] systemd[1]: Stopped target Local File Systems. gitlab # [ 4.917179] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 4.919198] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 4.921346] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 4.922893] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 4.924910] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 4.926859] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 4.928269] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 4.929999] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 4.931570] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 4.933388] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 4.934815] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 4.936465] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 4.937798] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 4.939231] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 4.940808] systemd[1]: systemd-udevd.service: Consumed 2.736s CPU time over 3.216s wall clock time, 30.5M memory peak. gitlab # [ 4.942988] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 4.944657] systemd[1]: Closed udev Control Socket. gitlab # [ 4.945728] systemd[1]: Starting Cleanup udev Database... gitlab # [ 4.946910] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 4.948471] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 4.949727] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 4.951344] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 4.952808] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 4.954198] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 5.019838] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 5.023158] systemd[1]: Finished Cleanup udev Database. gitlab # [ 5.025450] systemd[1]: Reached target Switch Root. gitlab # [ 5.027621] systemd[1]: Starting NixOS Activation... gitlab # [ 5.219614] initrd-nixos-activation-start[219]: booting system configuration /nix/store/abixm8krzs1bg4nv97nd774bz7iq4cfa-nixos-system-gitlab-test gitlab # [ 5.282093] initrd-nixos-activation-start[219]: running activation script... gitlab # [ 5.608690] initrd-nixos-activation-start[242]: setting up /etc... gitlab # [ 5.776356] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.779681] systemd[1]: Finished NixOS Activation. gitlab # [ 5.782049] systemd[1]: Starting Switch Root... gitlab # [ 5.827561] systemd[1]: Switching root. gitlab # [ 5.970144] systemd-journald[94]: Received SIGTERM from PID 1 (systemd). gitlab # [ 6.590422] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 6.590552] systemd[1]: Detected virtualization qemu. gitlab # [ 6.590625] systemd[1]: Detected architecture arm64. gitlab # [ 6.590793] systemd[1]: Detected first boot. gitlab # [ 6.592140] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.855683] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.973558] systemd[1]: Applying preset policy. gitlab # [ 7.194258] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 7.348738] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 7.349116] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 7.353433] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 7.360542] systemd[1]: Created slice Slice /system/getty. gitlab # [ 7.364260] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 7.368044] systemd[1]: Created slice User and Session Slice. gitlab # [ 7.369510] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 7.370831] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 7.372019] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 7.373055] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 7.374242] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 7.375361] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 7.376437] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 7.377526] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 7.378618] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 7.379763] systemd[1]: Reached target Path Units. gitlab # [ 7.380843] systemd[1]: Reached target Remote File Systems. gitlab # [ 7.381920] systemd[1]: Reached target Slice Units. gitlab # [ 7.382997] systemd[1]: Reached target Swaps. gitlab # [ 7.386946] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 7.391175] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 7.394270] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 7.397828] systemd[1]: Listening on Factory Reset Management. gitlab # [ 7.399132] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 7.408460] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 7.410324] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 7.413951] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 7.415457] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 7.416464] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.417391] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 7.422907] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 7.424269] systemd[1]: Listening on udev Control Socket. gitlab # [ 7.425496] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 7.465929] systemd[1]: Mounting Huge Pages File System... gitlab # [ 7.471608] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 7.478241] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 7.489496] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 7.499328] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 7.509602] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 7.514048] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.529598] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 7.533491] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 7.534241] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.549383] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 7.550677] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.562496] systemd[1]: Starting Journal Service... gitlab # [ 7.573650] systemd[1]: Starting Load Kernel Modules... gitlab # [ 7.581370] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 7.589746] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 7.590774] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.601463] fuse: init (API version 7.45) gitlab # [ 7.609361] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 7.615046] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 7.617885] systemd[1]: Mounted Huge Pages File System. gitlab # [ 7.621710] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 7.622668] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 7.623545] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 7.626956] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 7.628023] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 7.630495] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 7.631797] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 7.637151] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 7.638298] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 7.640039] systemd[1]: Finished Load Kernel Modules. gitlab # [ 7.650819] EXT4-fs (vda): re-mounted 8d4d7268-903c-4e87-aa46-8e462c808565. gitlab # [ 7.653380] systemd[1]: Mounting FUSE Control File System... gitlab # [ 7.659007] systemd[1]: Starting Firewall... gitlab # [ 7.675897] systemd-journald[313]: Collecting audit messages is enabled. gitlab # [ 7.688397] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 7.693556] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 7.689050] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 7.693771] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 7.695241] systemd[1]: systemd-journald.service: Deactivated successfully.[ 7.700683] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # gitlab # [ 7.706504] systemd[1]: Started Journal Service. gitlab # [ 7.704474] systemd-modules-load[314]: Using 3 probe threads gitlab # [ 7.705767] systemd-modules-load[314]: Module 'atkbd' is built in gitlab # [ 7.707616] systemd-modules-load[314]: Module 'loop' is built in gitlab # [ 7.708922] systemd-modules-load[314]: Inserted module 'tls' gitlab # [ 7.712752] systemd-oomd[315]: No swap; memory pressure usage will be degraded gitlab # [ 7.714641] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 7.715996] systemd[1]: Mounted FUSE Control File System. gitlab # [ 7.728301] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 7.737710] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 7.740366] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 7.743680] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.764876] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 7.772206] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 7.774280] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.782283] systemd-journald[313]: Received client request to flush runtime journal. gitlab # [ 7.813969] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 7.815673] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 7.818496] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 7.820777] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 7.822480] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 7.823514] systemd[1]: Reached target First Boot Complete. gitlab # [ 7.824918] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 7.865397] systemd-udevd[363]: Using default interface naming scheme 'v261'. gitlab # [ 7.898917] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 8.186251] systemd[1]: Finished Firewall. gitlab # [ 8.280600] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 8.352206] systemd[1]: Mounting /run/wrappers... gitlab # [ 8.364103] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 8.412953] systemd[1]: Mounted /run/wrappers. gitlab # [ 8.414620] systemd[1]: Reached target Local File Systems. gitlab # [ 8.419495] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 8.426609] systemd[1]: Starting register-nix-paths.service... gitlab # [ 8.431872] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 8.436475] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 8.462199] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 8.466881] systemd[1]: Starting Create System Files and Directories... gitlab # [ 8.480159] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 8.503014] (udev-worker)[471]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 8.510765] (udev-worker)[471]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 8.515802] (udev-worker)[464]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 8.519462] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 8.523075] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 8.525286] systemd[1]: Finished Create System Files and Directories. gitlab # [ 8.562876] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 8.567308] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 8.572259] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 8.573834] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 8.575405] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 8.579464] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 8.581028] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 8.584297] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 8.586745] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 8.591765] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 8.616937] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 8.628800] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 8.632335] systemd[1]: Starting Update is Completed... gitlab # [ 8.663482] mousedev: PS/2 mouse device common for all mice gitlab # [ 8.665583] systemd[1]: Finished Update is Completed. gitlab # [ 8.697734] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 8.698964] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 8.699847] systemd[1]: suid-sgid-wrappers.service: Consumed 164ms CPU time over 265ms wall clock time, 6.2M memory peak, 1.9M read from disk. gitlab # [ 8.941193] systemd[1]: Finished register-nix-paths.service. gitlab # [ 8.942473] systemd[1]: Reached target System Initialization. gitlab # [ 8.943372] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 8.944441] systemd[1]: Started logrotate.timer. gitlab # [ 8.945168] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 8.946400] systemd[1]: Reached target Timer Units. gitlab # [ 8.947169] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 8.949378] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 8.958003] systemd[1]: Starting nixbot CI service socket... gitlab # [ 8.960495] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 8.962644] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 8.964059] systemd[1]: Reached target Socket Units. gitlab # [ 8.966169] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 8.999370] dbus-broker-launch[557]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 9.001061] dbus-broker-launch[557]: NSS returned no entry for 'systemd-timesync' gitlab # [ 9.002701] dbus-broker-launch[557]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 9.006783] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 9.017674] dbus-broker-launch[557]: Ready gitlab # [ 9.018983] systemd[1]: Reached target Basic System. gitlab # [ 9.021132] systemd[1]: Started backdoor.service. gitlab # [ 9.024105] systemd[1]: Starting gitlab-config.service... gitlab # [ 9.046471] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 9.047966] systemd[1]: Starting Logrotate configuration check... gitlab # [ 9.052466] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 9.059861] systemd[1]: Starting Post-Boot Actions... gitlab # [ 9.062074] systemd[1]: Started Reset console on configuration changes. gitlab # [ 9.072138] systemd[1]: Starting resolvconf update... gitlab # [ 9.073634] logrotate[562]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 9.077769] logrotate[562]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 9.079671] logrotate[562]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 9.080831] logrotate[562]: Reading state from file: /var/lib/logrotate.status gitlab # [ 9.084327] logrotate[562]: state file /var/lib/logrotate.status does not exist gitlab # [ 9.085436] logrotate[562]: Allocating hash table for state file, size 64 entries gitlab # [ 9.086472] logrotate[562]: Handling 4 logs gitlab # [ 9.087089] logrotate[562]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 9.089199] logrotate[562]: considering log /var/log/btmp gitlab # [ 9.091126] logrotate[562]: Creating new state gitlab # [ 9.091765] logrotate[562]: Now: 2026-09-13 10:45 gitlab # [ 9.092528] logrotate[562]: Last rotated at 2026-09-13 10:00 gitlab # [ 9.093350] logrotate[562]: log does not need rotating (log has already been rotated) gitlab # [ 9.096142] logrotate[562]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 9.098345] logrotate[562]: considering log /var/log/wtmp gitlab # [ 9.099138] logrotate[562]: Creating new state gitlab # connecting to host... gitlab # [ 9.099950] logrotate[562]: Now: 2026-09-13 10:45 gitlab # [ 9.102339] logrotate[562]: Last rotated at 2026-09-13 10:00 gitlab # [ 9.103301] logrotate[562]: log does not need rotating (log has already been rotated) gitlab # [ 9.104459] logrotate[562]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 9.106428] logrotate[562]: switching euid from 0 to 165 and egid from 0 to 165 (pid 562) gitlab # [ 9.107588] logrotate[562]: considering log /var/gitlab/state/log/*.log gitlab # [ 9.109104] logrotate[562]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 9.111907] logrotate[562]: Creating new state gitlab # [ 9.112612] logrotate[562]: switching euid from 165 to 0 and egid from 165 to 0 (pid 562) gitlab # [ 9.114608] logrotate[562]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 9.120129] logrotate[562]: switching euid from 0 to 60 and egid from 0 to 60 (pid 562) gitlab # [ 9.121191] logrotate[562]: considering log /var/log/nginx/*.log gitlab # [ 9.121981] logrotate[562]: log /var/log/nginx/*.log does not exist -- skipping gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 9.124500] logrotate[562]: Creating new state gitlab # [ 9.125127] logrotate[562]: not running postrotate script, since no logs were rotated gitlab # [ 9.126188] logrotate[562]: switching euid from 60 to 0 and egid from 60 to 0 (pid 562) gitlab: connected to guest root shell gitlab # [ 9.129111] nsncd[563]: Sep 13 10:45:33.988 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab: (connecting took 9.60 seconds) gitlab # [ 9.131052] systemd[1]: Finished Logrotate configuration check. gitlab: (finished: waiting for the VM to finish booting, in 9.60 seconds) gitlab # [ 9.133610] systemd[1]: Finished Post-Boot Actions. gitlab # [ 9.134464] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 9.135369] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 9.138906] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 9.141101] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 9.142168] systemd[1]: Starting User Login Management... gitlab # [ 9.184204] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 9.187169] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 9.189243] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 9.190162] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 9.190980] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 9.194720] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 9.195595] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 9.196566] systemd[1]: nscd.service: Consumed 20ms CPU time over 142ms wall clock time, 7.6M memory peak, 5.8M read from disk. gitlab # [ 9.201198] systemd-logind[601]: New seat seat0. gitlab # [ 9.355723] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 9.234582] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 9.382744] systemd[1]: Finished resolvconf update. gitlab # [ 9.383586] systemd[1]: Reached target Preparation for Network. gitlab # [ 9.384797] systemd[1]: Starting DHCP Client... gitlab # [ 9.398024] nsncd[664]: Sep 13 10:45:34.161 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 9.402644] systemd[1]: Starting Address configuration of eth1... gitlab # [ 9.403515] systemd[1]: Starting Extra networking commands.... gitlab # [ 9.404361] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 9.405237] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 9.406097] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 9.407817] systemd-logind[601]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 9.409732] systemd-logind[601]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 9.410930] systemd[1]: Started User Login Management. gitlab # [ 9.430001] systemd[1]: Starting linger-users.service... gitlab # [ 9.449031] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 9.450005] systemd[1]: Finished linger-users.service. gitlab # [ 9.476734] network-addresses-eth1-start[679]: adding address 192.168.1.1/24... done gitlab # [ 9.484921] network-addresses-eth1-start[679]: adding address 2001:db8:1::1/64... done gitlab # [ 9.492676] systemd[1]: Starting Virtual Console Setup... gitlab # [ 9.508998] systemd[1]: Finished Address configuration of eth1. gitlab # [ 9.520853] systemd-logind[601]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 9.545923] systemd[1]: Finished Extra networking commands.. gitlab # [ 9.547713] systemd[1]: Reached target Network. gitlab # [ 9.552426] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 9.558144] systemd[1]: Starting Nginx Web Server... gitlab # [ 9.560331] dhcpcd[695]: dhcpcd-10.3.2 starting gitlab # [ 9.563642] dhcpcd[754]: dev: loaded udev gitlab # [ 9.567099] systemd[1]: Starting PostgreSQL Server... gitlab # [ 9.572437] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 9.575102] systemd-vconsole-setup[710]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 9.576999] systemd[1]: Starting Permit User Sessions... gitlab # [ 9.585561] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 9.585950] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 9.581346] systemd[1]: Finished Virtual Console Setup. gitlab # [ 9.596221] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 9.619382] systemd[1]: Finished Permit User Sessions. gitlab # [ 9.624938] systemd[1]: Started Getty on tty1. gitlab # [ 9.626977] systemd[1]: Reached target Login Prompts. gitlab # [ 9.635753] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 9.648460] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 9.648977] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 9.650710] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 9.651068] cfg80211: failed to load regulatory.db gitlab # [ 9.693726] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 9.691098] dhcpcd[754]: eth0: waiting for carrier gitlab # [ 9.691960] dhcpcd[754]: eth0: carrier acquired gitlab # [ 9.702593] dhcpcd[754]: DUID 00:01:00:01:32:39:3b:4e:52:54:00:12:34:56 gitlab # [ 9.703714] dhcpcd[754]: eth0: IAID 00:12:34:56 gitlab # [ 9.708102] dhcpcd[754]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 9.734407] nginx-pre-start[784]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 9.736232] nginx-pre-start[784]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 9.740455] systemd[1]: Started Nginx Web Server. gitlab # [ 9.765215] postgresql-pre-start[788]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 9.767942] postgresql-pre-start[788]: This user must also own the server process. gitlab # [ 9.770224] redis-server[783]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 9.772987] postgresql-pre-start[788]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 9.774284] postgresql-pre-start[788]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 9.775686] postgresql-pre-start[788]: The default text search configuration will be set to "english". gitlab # [ 9.777015] postgresql-pre-start[788]: Data page checksums are enabled. gitlab # [ 9.777958] postgresql-pre-start[788]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 9.779231] postgresql-pre-start[788]: creating subdirectories ... ok gitlab # [ 9.780371] postgresql-pre-start[788]: selecting dynamic shared memory implementation ... posix gitlab # [ 9.781554] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 9.782379] redis-server[783]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 9.783770] redis-server[783]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=783, just started gitlab # [ 9.785343] redis-server[783]: Configuration loaded gitlab # [ 9.786352] redis[783]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 9.787299] redis[783]: Running mode=standalone, port=0. gitlab # [ 9.788178] redis[783]: Server initialized gitlab # [ 9.788871] redis[783]: Ready to accept connections unix gitlab # [ 9.789771] redis[783]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 9.812111] postgresql-pre-start[788]: selecting default "max_connections" ... 100 gitlab # [ 9.863945] postgresql-pre-start[788]: selecting default "shared_buffers" ... 128MB gitlab # [ 10.181725] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 10.394760] postgresql-pre-start[788]: selecting default time zone ... UTC gitlab # [ 10.397436] postgresql-pre-start[788]: creating configuration files ... ok gitlab # [ 10.506988] gitlab-workhorse[815]: time="2026-09-13T10:45:35Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 10.511186] gitlab-workhorse[815]: time="2026-09-13T10:45:35Z" level=info msg="Using redis/go-redis" gitlab # [ 10.517378] gitlab-workhorse[815]: time="2026-09-13T10:45:35Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 10.651437] postgresql-pre-start[788]: running bootstrap script ... ok gitlab # [ 11.334596] dhcpcd[754]: eth0: soliciting a DHCP lease gitlab # [ 11.353404] dhcpcd[754]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 11.360561] postgresql-pre-start[788]: performing post-bootstrap initialization ... ok gitlab # [ 11.371211] dhcpcd[754]: eth0: probing address 10.0.2.15/24 gitlab # [ 11.672973] postgresql-pre-start[788]: syncing data to disk ... ok gitlab # [ 11.675707] postgresql-pre-start[788]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 11.679806] postgresql-pre-start[788]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 11.686370] postgresql-pre-start[788]: Success. You can now start the database server using: gitlab # [ 11.689932] postgresql-pre-start[788]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 11.776188] dhcpcd[754]: eth0: soliciting an IPv6 router gitlab # [ 11.778799] dhcpcd[754]: eth0: Router Advertisement from fe80::2 gitlab # [ 11.781395] dhcpcd[754]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 11.784294] dhcpcd[754]: eth0: adding route to fec0::/64 gitlab # [ 11.786547] dhcpcd[754]: eth0: adding default route via fe80::2 gitlab # [ 11.883456] postgres[849]: [849] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 11.889094] postgres[849]: [849] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 11.892392] postgres[849]: [849] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 11.895991] postgres[849]: [849] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 11.905533] postgres[858]: [858] LOG: database system was shut down at 2026-09-13 10:45:36 GMT gitlab # [ 11.917416] postgres[849]: [849] LOG: database system is ready to accept connections gitlab # [ 11.922080] systemd[1]: Started PostgreSQL Server. gitlab # [ 11.930219] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 12.248567] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[867]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 12.253098] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[867]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 12.341399] postgresql-setup-start[874]: CREATE DATABASE gitlab # [ 12.359536] systemd[1]: Finished gitlab-config.service. gitlab # [ 12.366168] systemd[1]: Started gitaly.service. gitlab # [ 12.398216] postgresql-setup-start[892]: CREATE ROLE gitlab # [ 12.424521] postgresql-setup-start[895]: ALTER DATABASE gitlab # [ 12.482396] postgresql-setup-start[904]: CREATE ROLE gitlab # [ 12.492394] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 12.494722] systemd[1]: Reached target PostgreSQL. gitlab # [ 12.504448] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 12.512151] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 12.711306] gitlab-postgresql-start[919]: CREATE DATABASE gitlab # [ 12.790188] gitlab-postgresql-start[930]: CREATE EXTENSION gitlab # [ 12.896579] gitlab-postgresql-start[932]: CREATE EXTENSION gitlab # [ 12.900756] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 13.080102] gitaly[891]: time="2026-09-13T10:45:37.954Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=891 gitlab # [ 13.083794] gitaly[891]: time="2026-09-13T10:45:37.958Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=891 gitlab # [ 13.088248] gitaly[891]: time="2026-09-13T10:45:37.958Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=891 gitlab # [ 13.090348] gitaly[891]: time="2026-09-13T10:45:37.958Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=891 gitlab # [ 13.096184] gitaly[891]: time="2026-09-13T10:45:37.958Z" level=info msg="Starting Gitaly" pid=891 version= gitlab # [ 13.316933] gitaly[891]: time="2026-09-13T10:45:38.192Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=233 pid=891 gitlab # [ 13.318721] gitaly[891]: time="2026-09-13T10:45:38.193Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=891 gitlab # [ 13.320209] gitaly[891]: time="2026-09-13T10:45:38.193Z" level=info msg="finished initializing command factory" duration_ms=0 pid=891 gitlab # [ 13.321748] gitaly[891]: time="2026-09-13T10:45:38.193Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-891/git-exec-1735017606.d/git pid=891 gitlab # [ 13.328494] gitaly[891]: time="2026-09-13T10:45:38.203Z" level=info msg="finished detecting git version" duration_ms=9 pid=891 gitlab # [ 13.330165] gitaly[891]: time="2026-09-13T10:45:38.203Z" level=info msg="using Git version" pid=891 version=2.53.g gitlab # [ 13.331588] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="clearing disk cache object folder" pid=891 storage=default gitlab # [ 13.333208] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1395676515 pid=891 storage=default gitlab # [ 13.335489] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1395676515 pid=891 storage=default gitlab # [ 13.338254] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=891 gitlab # [ 13.340720] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=891 gitlab # [ 13.343443] gitaly[891]: time="2026-09-13T10:45:38.204Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache1395676515 clear_duration_ms=0 pid=891 storage=default gitlab # [ 13.346311] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="counting repositories" pid=891 gitlab # [ 13.347517] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="starting to count repositories" pid=891 storage_path=/var/gitlab/state/repositories gitlab # [ 13.349389] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=891 gitlab # [ 13.351365] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=891 schema=unix gitlab # [ 13.353311] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="finished tempdir cleaner walk" pid=891 storage=default time_ms=0 gitlab # [ 13.354851] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="listening at address" address=/run/gitaly/gitaly-891/sock.d/intern pid=891 schema=unix gitlab # [ 13.356676] gitaly[891]: time="2026-09-13T10:45:38.219Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=891 storage_path=/var/gitlab/state/repositories gitlab # [ 13.358648] gitaly[891]: time="2026-09-13T10:45:38.220Z" level=info msg="completed counting all repositories" counting_duration=0 pid=891 gitlab # [ 13.360234] gitaly[891]: time="2026-09-13T10:45:38.221Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=891 gitlab # [ 13.361753] gitaly[891]: time="2026-09-13T10:45:38.221Z" level=info msg="Started Gitaly" duration_ms=263 pid=891 gitlab # [ 13.363038] gitaly[891]: time="2026-09-13T10:45:38.221Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=891 gitlab # [ 13.364610] gitaly[891]: time="2026-09-13T10:45:38.221Z" level=info msg="maintenance: daily scheduled" pid=891 scheduled="2026-09-13 12:00:00 +0000 UTC" gitlab # [ 13.974372] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.984765] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.994856] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 16.291331] dhcpcd[754]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 16.291687] dhcpcd[754]: eth0: adding route to 10.0.2.0/24 gitlab # [ 16.291913] dhcpcd[754]: eth0: adding default route via 10.0.2.2 gitlab # [ 16.513025] systemd[1]: Started DHCP Client. gitlab # [ 16.515038] systemd[1]: Reached target Network is Online. gitlab # [ 16.782238] gitaly[891]: time="2026-09-13T10:45:41.656Z" level=info msg="License database preloaded" duration_ms=3698 pid=891 gitlab # [ 60.446801] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[917]: W, [2026-09-13T10:46:25.320404 #917] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 60.583631] gitlab-workhorse[815]: time="2026-09-13T10:46:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 63.380905] postgres[1442]: [1442] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 63.381291] postgres[1442]: [1442] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 63.381437] postgres[1442]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 63.381556] postgres[1442]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 63.381680] postgres[1442]: attidentity AS identity, gitlab # [ 63.381811] postgres[1442]: attgenerated as attgenerated gitlab # [ 63.381928] postgres[1442]: FROM pg_attribute a gitlab # [ 63.382077] postgres[1442]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 63.382197] postgres[1442]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 63.382314] postgres[1442]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 63.382433] postgres[1442]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 63.382549] postgres[1442]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 63.382666] postgres[1442]: ORDER BY a.attnum gitlab # [ 63.382782] postgres[1442]: gitlab # [ 68.501270] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1493]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 68.501724] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1493]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 70.586122] gitlab-workhorse[815]: time="2026-09-13T10:46:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 80.589133] gitlab-workhorse[815]: time="2026-09-13T10:46:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 90.598430] gitlab-workhorse[815]: time="2026-09-13T10:46:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 95.730163] postgres[1606]: [1606] ERROR: relation "table_without_models" does not exist gitlab # [ 95.730656] postgres[1606]: [1606] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 100.618279] gitlab-workhorse[815]: time="2026-09-13T10:47:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 110.626524] gitlab-workhorse[815]: time="2026-09-13T10:47:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 120.637324] gitlab-workhorse[815]: time="2026-09-13T10:47:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 125.837189] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 125.843211] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 125.849662] systemd[1]: Starting gitlab.service... gitlab # [ 126.495257] bundle[1981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 126.501667] bundle[1981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 126.507554] bundle[1981]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 130.638291] gitlab-workhorse[815]: time="2026-09-13T10:47:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 131.347645] sidekiq-cluster[1980]: {"severity":"INFO","time":"2026-09-13T10:47:36.191Z","message":"Starting cluster with 1 processes"} gitlab # [ 131.472142] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.315Z","pid":1981,"message":"* Enabling systemd notification integration"} gitlab # [ 131.477556] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"Puma starting in cluster mode..."} gitlab # [ 131.482204] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 131.487482] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 131.493921] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Min threads: 0"} gitlab # [ 131.498272] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Max threads: 4"} gitlab # [ 131.502590] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Environment: production"} gitlab # [ 131.507064] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Master PID: 1981"} gitlab # [ 131.511214] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Workers: 2"} gitlab # [ 131.515170] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 131.520244] bundle[1981]: {"timestamp":"2026-09-13T10:47:36.346Z","pid":1981,"message":"* Preloading application"} gitlab # [ 140.646279] gitlab-workhorse[815]: time="2026-09-13T10:47:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 150.656924] gitlab-workhorse[815]: time="2026-09-13T10:47:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 160.666367] gitlab-workhorse[815]: time="2026-09-13T10:48:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 170.667464] gitlab-workhorse[815]: time="2026-09-13T10:48:15Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 170.975987] bundle[1981]: W, [2026-09-13T10:48:15.850167 #1981] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 174.492778] sidekiq-cluster[2030]: 2026-09-13T10:48:19.366Z pid=2030 tid=wna INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2030, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433119232} gitlab # [ 174.780234] sidekiq-cluster[2030]: W, [2026-09-13T10:48:19.655288 #2030] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 174.849012] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:19.724Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 179.545233] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.416Z","message":"GitLab reliable fetch activated!"} gitlab # [ 179.545565] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.417Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 179.621762] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.496Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 179.628785] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.503Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 179.635804] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.509Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 179.642303] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.516Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 179.649074] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.521Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 179.654454] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.527Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 179.659488] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.532Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 179.663799] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.536Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 179.667910] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.540Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 179.671795] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.543Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 179.675932] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.547Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 179.679705] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.551Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 179.683030] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.555Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 179.686402] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.558Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 179.689584] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.561Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 179.692663] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.565Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 179.695387] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.569Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 179.698121] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.573Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 179.701899] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.577Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 179.705643] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.580Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 179.709099] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.584Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 179.712709] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.588Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 179.716232] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.590Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 179.719878] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.593Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 179.723827] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.596Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 179.727567] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.599Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 179.731227] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.603Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 179.734831] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.606Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 179.738347] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.609Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 179.741780] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.613Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 179.744797] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.616Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 179.747940] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.619Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 179.750878] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.622Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 179.753822] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.626Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 179.756567] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.628Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 179.759130] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.631Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 179.761846] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.633Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 179.764310] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.635Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 179.766722] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.637Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 179.769128] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.639Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 179.771560] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.642Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 179.774044] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.644Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 179.776430] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.646Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 179.778596] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.648Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 179.780965] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.651Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 179.783348] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.653Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 179.785692] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.655Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 179.787853] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.657Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 179.790069] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.660Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 179.792138] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.662Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 179.794245] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.664Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 179.796299] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.667Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 179.805425] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.680Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 179.808557] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.683Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 179.811709] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.687Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 179.814690] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.689Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 179.818558] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.692Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 179.821614] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.694Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 179.824433] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.697Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 179.827253] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.699Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 179.829889] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.701Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 179.832398] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.703Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 179.834932] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.706Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 179.837658] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.708Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 179.840394] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.711Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 179.842881] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.713Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 179.845325] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.715Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 179.847721] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.717Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 179.850348] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.718Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 179.852820] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.720Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 179.855069] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.722Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 179.857401] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.724Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 179.859589] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.726Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 179.862049] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.727Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 179.864457] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.729Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 179.866764] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.731Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 179.869005] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.733Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 179.871399] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.735Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 179.873648] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.737Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 179.875710] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.738Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 179.878146] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.740Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 179.880603] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.742Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 179.882881] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.744Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 179.885128] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.746Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 179.887431] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.748Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 179.889731] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.750Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 179.892049] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.752Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 179.894273] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.754Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 179.896572] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.756Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 179.898807] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.758Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 179.901127] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.760Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 179.903279] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.762Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 179.905575] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.763Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 179.907884] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.765Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 179.910047] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.768Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 179.912305] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.771Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 179.914568] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.773Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 179.916891] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.775Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 179.919083] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.785Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 179.921375] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.788Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 179.923525] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.790Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 179.925895] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.793Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 179.927912] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.796Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 179.930213] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.799Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 179.932588] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.801Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 179.934888] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.804Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 179.937071] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.806Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 179.975337] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.848Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 179.978560] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:48:24.853Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 180.674283] gitlab-workhorse[815]: time="2026-09-13T10:48:25Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 190.675876] gitlab-workhorse[815]: time="2026-09-13T10:48:35Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 200.678263] gitlab-workhorse[815]: time="2026-09-13T10:48:45Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 210.686419] gitlab-workhorse[815]: time="2026-09-13T10:48:55Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 220.132678] postgres[2777]: [2777] ERROR: relation "table_without_models" does not exist gitlab # [ 220.133070] postgres[2777]: [2777] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 220.687605] gitlab-workhorse[815]: time="2026-09-13T10:49:05Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 223.508653] postgres[2806]: [2806] ERROR: relation "table_without_models" does not exist gitlab # [ 223.508986] postgres[2806]: [2806] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 225.085358] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:09.959Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 225.089327] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:09.964Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 225.089896] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:09.965Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 225.090157] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:09.965Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 225.092808] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:09.968Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 227.814483] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:12.688Z","message":"Cleaning working queues"} gitlab # [ 227.816231] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:12.690Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0eddcefd95cb9a9f727803cd","created_at":"2026-09-13T10:47:30.122Z","trace_propagation_headers":{"sentry-trace":"a636bc994bc94bf4b701730720da0210-8f0de12c4feb4064","baggage":"sentry-trace_id=a636bc994bc94bf4b701730720da0210,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"372a2c4c3d0da5cf007964c4c627d27d","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:47:30.133Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0eddcefd95cb9a9f727803cd: start","job_status":"start","queue_duration_s":102.556384,"scheduling_latency_s":102.556436} gitlab # [ 228.072308] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:12.946Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"0eddcefd95cb9a9f727803cd","created_at":"2026-09-13T10:47:30.122Z","trace_propagation_headers":{"sentry-trace":"a636bc994bc94bf4b701730720da0210-8f0de12c4feb4064","baggage":"sentry-trace_id=a636bc994bc94bf4b701730720da0210,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"372a2c4c3d0da5cf007964c4c627d27d","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:47:30.133Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-0eddcefd95cb9a9f727803cd: done: 0.256696 sec","job_status":"done","queue_duration_s":102.556384,"scheduling_latency_s":102.556436,"redis_calls":29,"redis_duration_s":0.010076,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.006183,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002389,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001504,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.109001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14651236,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.256696,"completed_at":"2026-09-13T10:49:12.946Z","load_balancing_strategy":"primary","db_duration_s":0.099551,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 229.227618] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.101Z","pid":1981,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 229.227966] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.101Z","pid":1981,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 229.228224] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.101Z","pid":1981,"message":"! #\u003cThread:0x0000e574f4642b58 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 229.228413] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.102Z","pid":1981,"message":"! #\u003cThread:0x0000e574f1fcad68 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 229.228533] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.102Z","pid":1981,"message":"Use Ctrl-C to stop"} gitlab # [ 229.228647] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.102Z","pid":1981,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 229.872537] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.747Z","pid":1981,"message":"- Worker 1 (PID: 2919) booted in 0.59s, phase: 0"} gitlab # [ 229.878219] bundle[1981]: {"timestamp":"2026-09-13T10:49:14.747Z","pid":1981,"message":"- Worker 0 (PID: 2915) booted in 0.61s, phase: 0"} gitlab # [ 229.884136] systemd[1]: Started gitlab.service. gitlab # [ 229.886296] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 229.891393] systemd[1]: Starting setup-gitlab.service... gitlab # [ 229.941068] setup-gitlab-start[2952]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 233.462837] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:49:18 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3481 gitlab # [ 233.468509] setup-gitlab-start[2952]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 233.471646] setup-gitlab-start[2952]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 233.488378] runuser[2979]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 234.057108] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 234.063663] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 234.069968] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 234.886658] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 234.886981] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 234.889692] setup-gitlab-start[2980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 238.111719] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:22.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"66cbccec53b753416d29a908","created_at":"2026-09-13T10:49:22.873Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:22.981Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-66cbccec53b753416d29a908: start","job_status":"start","queue_duration_s":0.004116,"scheduling_latency_s":0.00417} gitlab # [ 238.918694] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:23.792Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"66cbccec53b753416d29a908","created_at":"2026-09-13T10:49:22.873Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:22.981Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-66cbccec53b753416d29a908: done: 0.807046 sec","job_status":"done","queue_duration_s":0.004116,"scheduling_latency_s":0.00417,"redis_calls":71,"redis_duration_s":0.29548800000000003,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.029648,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002509,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.257738,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.303524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00298942,"duration_s":0.807046,"completed_at":"2026-09-13T10:49:23.792Z","load_balancing_strategy":"primary","db_duration_s":0.029316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.015346] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:23.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c1af6378aa302fb67ed5f6c4","created_at":"2026-09-13T10:49:23.027Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"979af386fcf3f0fb862f55d4b03de4fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.042Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c1af6378aa302fb67ed5f6c4: start","job_status":"start","queue_duration_s":0.784771,"scheduling_latency_s":0.784862} gitlab # [ 239.275097] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.148Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"c1af6378aa302fb67ed5f6c4","created_at":"2026-09-13T10:49:23.027Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"979af386fcf3f0fb862f55d4b03de4fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.042Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-c1af6378aa302fb67ed5f6c4: done: 0.320908 sec","job_status":"done","queue_duration_s":0.784771,"scheduling_latency_s":0.784862,"redis_calls":14,"redis_duration_s":0.006609,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001786,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00201,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.044,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.138733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0013202,"exclusive_lock_hold_duration_s":0.25383064000000005,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.320908,"completed_at":"2026-09-13T10:49:24.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051582,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.390034] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.151Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"81f07407d7a63a45dc5afd89","created_at":"2026-09-13T10:49:23.051Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35c0492dd7fc6b5c9c7a6507a26b9dbb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.078Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-81f07407d7a63a45dc5afd89: start","job_status":"start","queue_duration_s":1.072785,"scheduling_latency_s":1.072829} gitlab # [ 239.405192] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.201Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"81f07407d7a63a45dc5afd89","created_at":"2026-09-13T10:49:23.051Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"35c0492dd7fc6b5c9c7a6507a26b9dbb","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.078Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-81f07407d7a63a45dc5afd89: done: 0.049562 sec","job_status":"done","queue_duration_s":1.072785,"scheduling_latency_s":1.072829,"redis_calls":12,"redis_duration_s":0.004233,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000766,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001608,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00109076,"exclusive_lock_hold_duration_s":0.00745342,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.049562,"completed_at":"2026-09-13T10:49:24.200Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.449693] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"672f8811f3c1ad62e073550e","created_at":"2026-09-13T10:49:23.129Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"085eb02b100546d84d08aa172f2bd941","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.140Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-672f8811f3c1ad62e073550e: start","job_status":"start","queue_duration_s":1.063887,"scheduling_latency_s":1.063922} gitlab # [ 239.464224] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.225Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 239.467756] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.237Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"672f8811f3c1ad62e073550e","created_at":"2026-09-13T10:49:23.129Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"085eb02b100546d84d08aa172f2bd941","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.140Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-672f8811f3c1ad62e073550e: done: 0.033682 sec","job_status":"done","queue_duration_s":1.063887,"scheduling_latency_s":1.063922,"redis_calls":10,"redis_duration_s":0.0036179999999999997,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00117,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000627,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044822,"duration_s":0.033682,"completed_at":"2026-09-13T10:49:24.237Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000779,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.510253] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.240Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8079067044c2a06e00abd04f","created_at":"2026-09-13T10:49:23.146Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0bd1283f81bbe0b5c09793fd5fcd5a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.167Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8079067044c2a06e00abd04f: start","job_status":"start","queue_duration_s":1.072274,"scheduling_latency_s":1.072302} gitlab # [ 239.524695] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.258Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 239.527767] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.269Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8079067044c2a06e00abd04f","created_at":"2026-09-13T10:49:23.146Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a0bd1283f81bbe0b5c09793fd5fcd5a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.167Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8079067044c2a06e00abd04f: done: 0.029431 sec","job_status":"done","queue_duration_s":1.072274,"scheduling_latency_s":1.072302,"redis_calls":10,"redis_duration_s":0.002938,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000697,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001621,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016872,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039656,"duration_s":0.029431,"completed_at":"2026-09-13T10:49:24.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.569790] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.272Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5214d7409bc7e880a9568241","created_at":"2026-09-13T10:49:23.198Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c28c2ebfeeb8e7821ed9338c61785152","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.225Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5214d7409bc7e880a9568241: start","job_status":"start","queue_duration_s":1.046652,"scheduling_latency_s":1.046702} gitlab # [ 239.583872] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.361Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"5214d7409bc7e880a9568241","created_at":"2026-09-13T10:49:23.198Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c28c2ebfeeb8e7821ed9338c61785152","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.225Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-5214d7409bc7e880a9568241: done: 0.089877 sec","job_status":"done","queue_duration_s":1.046652,"scheduling_latency_s":1.046702,"redis_calls":16,"redis_duration_s":0.006045,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003228,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00082,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00064818,"duration_s":0.089877,"completed_at":"2026-09-13T10:49:24.361Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.625786] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.365Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3071d63d46f3df914ac64954","created_at":"2026-09-13T10:49:23.262Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f773dd154be3bddb29caa89647245a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.286Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3071d63d46f3df914ac64954: start","job_status":"start","queue_duration_s":1.07929,"scheduling_latency_s":1.079338} gitlab # [ 239.640843] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.389Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 239.644216] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3071d63d46f3df914ac64954","created_at":"2026-09-13T10:49:23.262Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f773dd154be3bddb29caa89647245a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.286Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3071d63d46f3df914ac64954: done: 0.039495 sec","job_status":"done","queue_duration_s":1.07929,"scheduling_latency_s":1.079338,"redis_calls":10,"redis_duration_s":0.002488,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000517,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043218,"duration_s":0.039495,"completed_at":"2026-09-13T10:49:24.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000813,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.686056] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.407Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"086bcdf2a1d908fd52b1514c","created_at":"2026-09-13T10:49:23.319Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91ab7da042ccd75d9636797036251708","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.344Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-086bcdf2a1d908fd52b1514c: start","job_status":"start","queue_duration_s":1.062831,"scheduling_latency_s":1.062897} gitlab # [ 239.700767] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.487Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"086bcdf2a1d908fd52b1514c","created_at":"2026-09-13T10:49:23.319Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91ab7da042ccd75d9636797036251708","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.344Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-086bcdf2a1d908fd52b1514c: done: 0.080431 sec","job_status":"done","queue_duration_s":1.062831,"scheduling_latency_s":1.062897,"redis_calls":12,"redis_duration_s":0.003167,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000415,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001505,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001247,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.000904,"exclusive_lock_hold_duration_s":0.03250578,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.080431,"completed_at":"2026-09-13T10:49:24.487Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.02833,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.745168] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.490Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"eb52830d6d579b883080911f","created_at":"2026-09-13T10:49:23.396Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"384b1e26857c1bd944d28a16954a4b18","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.419Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-eb52830d6d579b883080911f: start","job_status":"start","queue_duration_s":1.070369,"scheduling_latency_s":1.070404} gitlab # [ 239.955706] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.830Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"eb52830d6d579b883080911f","created_at":"2026-09-13T10:49:23.396Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"384b1e26857c1bd944d28a16954a4b18","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.419Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-eb52830d6d579b883080911f: done: 0.340349 sec","job_status":"done","queue_duration_s":1.070369,"scheduling_latency_s":1.070404,"redis_calls":565,"redis_duration_s":0.062053000000000004,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001046,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.060365,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.309497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002737,"duration_s":0.340349,"completed_at":"2026-09-13T10:49:24.830Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000673,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.005798] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.832Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"19d0ad7f3ab2909fadbd8a8d","created_at":"2026-09-13T10:49:23.449Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20f4f1a8b7203b8886d0ae4790340c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.474Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-19d0ad7f3ab2909fadbd8a8d: start","job_status":"start","queue_duration_s":1.357565,"scheduling_latency_s":1.357584} gitlab # [ 240.020267] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"19d0ad7f3ab2909fadbd8a8d","created_at":"2026-09-13T10:49:23.449Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d20f4f1a8b7203b8886d0ae4790340c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.474Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-19d0ad7f3ab2909fadbd8a8d: done: 0.024468 sec","job_status":"done","queue_duration_s":1.357565,"scheduling_latency_s":1.357584,"redis_calls":12,"redis_duration_s":0.00324,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001062,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063892,"duration_s":0.024468,"completed_at":"2026-09-13T10:49:24.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.063533] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8efd06f6118282a7e9960587","created_at":"2026-09-13T10:49:23.495Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"080894659e8519b1878d52254c7f243c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.524Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8efd06f6118282a7e9960587: start","job_status":"start","queue_duration_s":1.333358,"scheduling_latency_s":1.333377} gitlab # [ 240.078121] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8efd06f6118282a7e9960587","created_at":"2026-09-13T10:49:23.495Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"080894659e8519b1878d52254c7f243c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.524Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8efd06f6118282a7e9960587: done: 0.020531 sec","job_status":"done","queue_duration_s":1.333358,"scheduling_latency_s":1.333377,"redis_calls":10,"redis_duration_s":0.002359,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000487,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001372,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033668,"duration_s":0.020531,"completed_at":"2026-09-13T10:49:24.878Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000682,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.121079] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.880Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c859002672a390f3fe2e7edf","created_at":"2026-09-13T10:49:23.552Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4f5ffe9cdf92f9791c8eea146b44a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.573Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c859002672a390f3fe2e7edf: start","job_status":"start","queue_duration_s":1.307581,"scheduling_latency_s":1.3076} gitlab # [ 240.134835] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.908Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"c859002672a390f3fe2e7edf","created_at":"2026-09-13T10:49:23.552Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4f5ffe9cdf92f9791c8eea146b44a9e","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.573Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-c859002672a390f3fe2e7edf: done: 0.02726 sec","job_status":"done","queue_duration_s":1.307581,"scheduling_latency_s":1.3076,"redis_calls":10,"redis_duration_s":0.002462,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000474,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001492,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01491,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035122,"duration_s":0.02726,"completed_at":"2026-09-13T10:49:24.908Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.176573] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.909Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7c5866069dfb73097125f20d","created_at":"2026-09-13T10:49:23.605Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eee9d825c38afe38bfab883418064ef9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:49:23.629Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7c5866069dfb73097125f20d: start","job_status":"start","queue_duration_s":1.279983,"scheduling_latency_s":1.280001} gitlab # [ 240.189902] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.922Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 240.193247] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"7c5866069dfb73097125f20d","created_at":"2026-09-13T10:49:23.605Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eee9d825c38afe38bfab883418064ef9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:49:23.629Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-7c5866069dfb73097125f20d: done: 0.020841 sec","job_status":"done","queue_duration_s":1.279983,"scheduling_latency_s":1.280001,"redis_calls":10,"redis_duration_s":0.002532,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00142,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011595,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032102,"duration_s":0.020841,"completed_at":"2026-09-13T10:49:24.930Z","load_balancing_strategy":"primary","db_duration_s":0.000786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.234331] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"398ae05f4f73d36ea19942b8","created_at":"2026-09-13T10:49:23.700Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.724Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-398ae05f4f73d36ea19942b8: start","job_status":"start","queue_duration_s":1.207746,"scheduling_latency_s":1.207765} gitlab # [ 240.249239] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.992Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"398ae05f4f73d36ea19942b8","created_at":"2026-09-13T10:49:23.700Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.724Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-398ae05f4f73d36ea19942b8: done: 0.059716 sec","job_status":"done","queue_duration_s":1.207746,"scheduling_latency_s":1.207765,"redis_calls":19,"redis_duration_s":0.016818,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003508,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000346,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.008853,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004111,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034518,"duration_s":0.059716,"completed_at":"2026-09-13T10:49:24.992Z","load_balancing_strategy":"primary","db_duration_s":0.00806,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.292195] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:24.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35f762353f74161bebc3e776","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35f762353f74161bebc3e776: start","job_status":"start","queue_duration_s":1.242315,"scheduling_latency_s":1.242333} gitlab # [ 240.304320] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35f762353f74161bebc3e776","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35f762353f74161bebc3e776: done: 0.023581 sec","job_status":"done","queue_duration_s":1.242315,"scheduling_latency_s":1.242333,"redis_calls":16,"redis_duration_s":0.004142,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002522,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032344,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023581,"completed_at":"2026-09-13T10:49:25.017Z","load_balancing_strategy":"primary","db_duration_s":0.000808,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.345362] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.018Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65c5f438a5b2be857affea9f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65c5f438a5b2be857affea9f: start","job_status":"start","queue_duration_s":1.267242,"scheduling_latency_s":1.26726} gitlab # [ 240.357822] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.035Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"65c5f438a5b2be857affea9f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-65c5f438a5b2be857affea9f: done: 0.017423 sec","job_status":"done","queue_duration_s":1.267242,"scheduling_latency_s":1.26726,"redis_calls":13,"redis_duration_s":0.003511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001473,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002038,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010245,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017423,"completed_at":"2026-09-13T10:49:25.035Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.396552] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ff80940043698063a3e82eb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ff80940043698063a3e82eb: start","job_status":"start","queue_duration_s":1.28606,"scheduling_latency_s":1.286077} gitlab # [ 240.408845] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ff80940043698063a3e82eb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ff80940043698063a3e82eb: done: 0.016702 sec","job_status":"done","queue_duration_s":1.28606,"scheduling_latency_s":1.286077,"redis_calls":13,"redis_duration_s":0.003527,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002576,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016702,"completed_at":"2026-09-13T10:49:25.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.448327] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.055Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902d42ae892c2e57305334fb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902d42ae892c2e57305334fb: start","job_status":"start","queue_duration_s":1.304145,"scheduling_latency_s":1.304164} gitlab # [ 240.460342] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"902d42ae892c2e57305334fb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-902d42ae892c2e57305334fb: done: 0.016539 sec","job_status":"done","queue_duration_s":1.304145,"scheduling_latency_s":1.304164,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016539,"completed_at":"2026-09-13T10:49:25.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.498964] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc20be48bbb60f88e2bd8155","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc20be48bbb60f88e2bd8155: start","job_status":"start","queue_duration_s":1.322243,"scheduling_latency_s":1.322261} gitlab # [ 240.511422] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc20be48bbb60f88e2bd8155","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc20be48bbb60f88e2bd8155: done: 0.015819 sec","job_status":"done","queue_duration_s":1.322243,"scheduling_latency_s":1.322261,"redis_calls":13,"redis_duration_s":0.0025989999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001712,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015819,"completed_at":"2026-09-13T10:49:25.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.550484] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"417f68c46b94fa2ddf3c7b36","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-417f68c46b94fa2ddf3c7b36: start","job_status":"start","queue_duration_s":1.339414,"scheduling_latency_s":1.339432} gitlab # [ 240.562892] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"417f68c46b94fa2ddf3c7b36","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-417f68c46b94fa2ddf3c7b36: done: 0.015605 sec","job_status":"done","queue_duration_s":1.339414,"scheduling_latency_s":1.339432,"redis_calls":13,"redis_duration_s":0.002681,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015605,"completed_at":"2026-09-13T10:49:25.106Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.603277] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba3b30fb900eaa569ae98a8","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba3b30fb900eaa569ae98a8: start","job_status":"start","queue_duration_s":1.356652,"scheduling_latency_s":1.356669} gitlab # [ 240.616028] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ba3b30fb900eaa569ae98a8","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ba3b30fb900eaa569ae98a8: done: 0.015915 sec","job_status":"done","queue_duration_s":1.356652,"scheduling_latency_s":1.356669,"redis_calls":13,"redis_duration_s":0.002689,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001767,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009884,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015915,"completed_at":"2026-09-13T10:49:25.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.655929] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ad33f7782584e11ee6278d4","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ad33f7782584e11ee6278d4: start","job_status":"start","queue_duration_s":1.374037,"scheduling_latency_s":1.374057} gitlab # [ 240.668542] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.140Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ad33f7782584e11ee6278d4","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ad33f7782584e11ee6278d4: done: 0.015225 sec","job_status":"done","queue_duration_s":1.374037,"scheduling_latency_s":1.374057,"redis_calls":13,"redis_duration_s":0.001534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015225,"completed_at":"2026-09-13T10:49:25.140Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.707353] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9119bbd26ff230d8d3b844ac","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9119bbd26ff230d8d3b844ac: start","job_status":"start","queue_duration_s":1.390487,"scheduling_latency_s":1.390506} gitlab # [ 240.719706] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9119bbd26ff230d8d3b844ac","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9119bbd26ff230d8d3b844ac: done: 0.015155 sec","job_status":"done","queue_duration_s":1.390487,"scheduling_latency_s":1.390506,"redis_calls":13,"redis_duration_s":0.0015469999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015155,"completed_at":"2026-09-13T10:49:25.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.758872] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.158Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86619f810f5f6f6edad464a9","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86619f810f5f6f6edad464a9: start","job_status":"start","queue_duration_s":1.406911,"scheduling_latency_s":1.40693} gitlab # [ 240.771129] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86619f810f5f6f6edad464a9","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86619f810f5f6f6edad464a9: done: 0.014709 sec","job_status":"done","queue_duration_s":1.406911,"scheduling_latency_s":1.40693,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014709,"completed_at":"2026-09-13T10:49:25.172Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.810307] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.174Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7038c5c81015df493c2d25b3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7038c5c81015df493c2d25b3: start","job_status":"start","queue_duration_s":1.423039,"scheduling_latency_s":1.423057} gitlab # [ 240.822913] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7038c5c81015df493c2d25b3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7038c5c81015df493c2d25b3: done: 0.014851 sec","job_status":"done","queue_duration_s":1.423039,"scheduling_latency_s":1.423057,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014851,"completed_at":"2026-09-13T10:49:25.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.862114] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"378bb24f2d2ba368794b95b6","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-378bb24f2d2ba368794b95b6: start","job_status":"start","queue_duration_s":1.438973,"scheduling_latency_s":1.438991} gitlab # [ 240.874380] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"378bb24f2d2ba368794b95b6","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-378bb24f2d2ba368794b95b6: done: 0.014982 sec","job_status":"done","queue_duration_s":1.438973,"scheduling_latency_s":1.438991,"redis_calls":13,"redis_duration_s":0.001593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014982,"completed_at":"2026-09-13T10:49:25.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.912638] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"affa56fd386bad2f941d9ae1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-affa56fd386bad2f941d9ae1: start","job_status":"start","queue_duration_s":1.455027,"scheduling_latency_s":1.455044} gitlab # [ 240.924721] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"affa56fd386bad2f941d9ae1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-affa56fd386bad2f941d9ae1: done: 0.014965 sec","job_status":"done","queue_duration_s":1.455027,"scheduling_latency_s":1.455044,"redis_calls":13,"redis_duration_s":0.001643,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014965,"completed_at":"2026-09-13T10:49:25.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.962863] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4013e2bb2d67dfe508ba7710","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4013e2bb2d67dfe508ba7710: start","job_status":"start","queue_duration_s":1.471033,"scheduling_latency_s":1.47105} gitlab # [ 240.975118] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4013e2bb2d67dfe508ba7710","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4013e2bb2d67dfe508ba7710: done: 0.015443 sec","job_status":"done","queue_duration_s":1.471033,"scheduling_latency_s":1.47105,"redis_calls":13,"redis_duration_s":0.00165,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015443,"completed_at":"2026-09-13T10:49:25.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.013639] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eadd4967a3c72d1cbbf4a833","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eadd4967a3c72d1cbbf4a833: start","job_status":"start","queue_duration_s":1.48762,"scheduling_latency_s":1.487638} gitlab # [ 241.025745] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eadd4967a3c72d1cbbf4a833","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eadd4967a3c72d1cbbf4a833: done: 0.015479 sec","job_status":"done","queue_duration_s":1.48762,"scheduling_latency_s":1.487638,"redis_calls":13,"redis_duration_s":0.001658,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010596,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015479,"completed_at":"2026-09-13T10:49:25.254Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.065454] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31dbd4f2c3aac764ae55a874","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31dbd4f2c3aac764ae55a874: start","job_status":"start","queue_duration_s":1.504225,"scheduling_latency_s":1.504243} gitlab # [ 241.077822] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31dbd4f2c3aac764ae55a874","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31dbd4f2c3aac764ae55a874: done: 0.014677 sec","job_status":"done","queue_duration_s":1.504225,"scheduling_latency_s":1.504243,"redis_calls":13,"redis_duration_s":0.0014429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014677,"completed_at":"2026-09-13T10:49:25.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.116696] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165708081189f3abc48224ac","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165708081189f3abc48224ac: start","job_status":"start","queue_duration_s":1.520129,"scheduling_latency_s":1.520147} gitlab # [ 241.129007] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"165708081189f3abc48224ac","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-165708081189f3abc48224ac: done: 0.015342 sec","job_status":"done","queue_duration_s":1.520129,"scheduling_latency_s":1.520147,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015342,"completed_at":"2026-09-13T10:49:25.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.168034] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcda687ee39f724844634c7b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcda687ee39f724844634c7b: start","job_status":"start","queue_duration_s":1.536777,"scheduling_latency_s":1.536795} gitlab # [ 241.180589] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bcda687ee39f724844634c7b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bcda687ee39f724844634c7b: done: 0.014655 sec","job_status":"done","queue_duration_s":1.536777,"scheduling_latency_s":1.536795,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014655,"completed_at":"2026-09-13T10:49:25.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.219927] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"411762988074d9030f02b12f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-411762988074d9030f02b12f: start","job_status":"start","queue_duration_s":1.552747,"scheduling_latency_s":1.552766} gitlab # [ 241.232589] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"411762988074d9030f02b12f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-411762988074d9030f02b12f: done: 0.014696 sec","job_status":"done","queue_duration_s":1.552747,"scheduling_latency_s":1.552766,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000501,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014696,"completed_at":"2026-09-13T10:49:25.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.272809] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e94865a0b86f5a341d2340eb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e94865a0b86f5a341d2340eb: start","job_status":"start","queue_duration_s":1.568787,"scheduling_latency_s":1.568805} gitlab # [ 241.285094] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e94865a0b86f5a341d2340eb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e94865a0b86f5a341d2340eb: done: 0.015659 sec","job_status":"done","queue_duration_s":1.568787,"scheduling_latency_s":1.568805,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01069,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015659,"completed_at":"2026-09-13T10:49:25.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.324216] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897047bf362aed5883cc7817","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897047bf362aed5883cc7817: start","job_status":"start","queue_duration_s":1.585738,"scheduling_latency_s":1.585757} gitlab # [ 241.336599] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"897047bf362aed5883cc7817","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-897047bf362aed5883cc7817: done: 0.01472 sec","job_status":"done","queue_duration_s":1.585738,"scheduling_latency_s":1.585757,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000888,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01472,"completed_at":"2026-09-13T10:49:25.351Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.375089] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37e2cd1656c64aab235179a8","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37e2cd1656c64aab235179a8: start","job_status":"start","queue_duration_s":1.601502,"scheduling_latency_s":1.601519} gitlab # [ 241.387618] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"37e2cd1656c64aab235179a8","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-37e2cd1656c64aab235179a8: done: 0.015331 sec","job_status":"done","queue_duration_s":1.601502,"scheduling_latency_s":1.601519,"redis_calls":13,"redis_duration_s":0.001468,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015331,"completed_at":"2026-09-13T10:49:25.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.426576] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c17390fc56c12f1c215898e","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c17390fc56c12f1c215898e: start","job_status":"start","queue_duration_s":1.617894,"scheduling_latency_s":1.617912} gitlab # [ 241.438992] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c17390fc56c12f1c215898e","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c17390fc56c12f1c215898e: done: 0.014911 sec","job_status":"done","queue_duration_s":1.617894,"scheduling_latency_s":1.617912,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010349,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014911,"completed_at":"2026-09-13T10:49:25.383Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.478808] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc3bbe886198a6decf67b34","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc3bbe886198a6decf67b34: start","job_status":"start","queue_duration_s":1.634066,"scheduling_latency_s":1.634083} gitlab # [ 241.491271] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc3bbe886198a6decf67b34","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc3bbe886198a6decf67b34: done: 0.014927 sec","job_status":"done","queue_duration_s":1.634066,"scheduling_latency_s":1.634083,"redis_calls":13,"redis_duration_s":0.001438,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-09-13T10:49:25.400Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.530129] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3151c0ec53a755d918844eb9","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3151c0ec53a755d918844eb9: start","job_status":"start","queue_duration_s":1.650094,"scheduling_latency_s":1.650113} gitlab # [ 241.542256] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3151c0ec53a755d918844eb9","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3151c0ec53a755d918844eb9: done: 0.015006 sec","job_status":"done","queue_duration_s":1.650094,"scheduling_latency_s":1.650113,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015006,"completed_at":"2026-09-13T10:49:25.416Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.580987] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.417Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ccde474ed0acf8e26c322d5","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ccde474ed0acf8e26c322d5: start","job_status":"start","queue_duration_s":1.666148,"scheduling_latency_s":1.666166} gitlab # [ 241.593196] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5ccde474ed0acf8e26c322d5","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5ccde474ed0acf8e26c322d5: done: 0.014646 sec","job_status":"done","queue_duration_s":1.666148,"scheduling_latency_s":1.666166,"redis_calls":13,"redis_duration_s":0.001455,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014646,"completed_at":"2026-09-13T10:49:25.431Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.632116] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4fb97b0307f39e34e0e5e5","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4fb97b0307f39e34e0e5e5: start","job_status":"start","queue_duration_s":1.682015,"scheduling_latency_s":1.682034} gitlab # [ 241.645311] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.448Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6b4fb97b0307f39e34e0e5e5","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6b4fb97b0307f39e34e0e5e5: done: 0.015441 sec","job_status":"done","queue_duration_s":1.682015,"scheduling_latency_s":1.682034,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010418,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015441,"completed_at":"2026-09-13T10:49:25.448Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.685184] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7081012f2a0cb8bf597bf024","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7081012f2a0cb8bf597bf024: start","job_status":"start","queue_duration_s":1.698664,"scheduling_latency_s":1.698681} gitlab # [ 241.697461] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.464Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7081012f2a0cb8bf597bf024","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7081012f2a0cb8bf597bf024: done: 0.014671 sec","job_status":"done","queue_duration_s":1.698664,"scheduling_latency_s":1.698681,"redis_calls":13,"redis_duration_s":0.001403,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014671,"completed_at":"2026-09-13T10:49:25.464Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.736365] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b5da2272911169f899af95f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b5da2272911169f899af95f: start","job_status":"start","queue_duration_s":1.714557,"scheduling_latency_s":1.714575} gitlab # [ 241.748514] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b5da2272911169f899af95f","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b5da2272911169f899af95f: done: 0.0148 sec","job_status":"done","queue_duration_s":1.714557,"scheduling_latency_s":1.714575,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010462,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0148,"completed_at":"2026-09-13T10:49:25.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.786962] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbdab3ce5a280cc9399d411d","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbdab3ce5a280cc9399d411d: start","job_status":"start","queue_duration_s":1.730576,"scheduling_latency_s":1.730594} gitlab # [ 241.799212] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dbdab3ce5a280cc9399d411d","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dbdab3ce5a280cc9399d411d: done: 0.015309 sec","job_status":"done","queue_duration_s":1.730576,"scheduling_latency_s":1.730594,"redis_calls":13,"redis_duration_s":0.001673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015309,"completed_at":"2026-09-13T10:49:25.497Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.837772] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.498Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042fae2eabdc588bc85a0752","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042fae2eabdc588bc85a0752: start","job_status":"start","queue_duration_s":1.74718,"scheduling_latency_s":1.747198} gitlab # [ 241.850147] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"042fae2eabdc588bc85a0752","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-042fae2eabdc588bc85a0752: done: 0.014716 sec","job_status":"done","queue_duration_s":1.74718,"scheduling_latency_s":1.747198,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014716,"completed_at":"2026-09-13T10:49:25.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.889624] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.514Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b27551bf403be6084bfb042","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b27551bf403be6084bfb042: start","job_status":"start","queue_duration_s":1.762969,"scheduling_latency_s":1.762987} gitlab # [ 241.901769] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.529Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b27551bf403be6084bfb042","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b27551bf403be6084bfb042: done: 0.014859 sec","job_status":"done","queue_duration_s":1.762969,"scheduling_latency_s":1.762987,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014859,"completed_at":"2026-09-13T10:49:25.529Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.940746] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b9243f64f4650d83f2970b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b9243f64f4650d83f2970b: start","job_status":"start","queue_duration_s":1.778845,"scheduling_latency_s":1.778861} gitlab # [ 241.952941] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.545Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73b9243f64f4650d83f2970b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73b9243f64f4650d83f2970b: done: 0.015486 sec","job_status":"done","queue_duration_s":1.778845,"scheduling_latency_s":1.778861,"redis_calls":13,"redis_duration_s":0.001665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000727,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-09-13T10:49:25.545Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.991750] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbd623f7e2c239cf36571001","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbd623f7e2c239cf36571001: start","job_status":"start","queue_duration_s":1.795395,"scheduling_latency_s":1.795412} gitlab # [ 242.004259] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.561Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbd623f7e2c239cf36571001","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbd623f7e2c239cf36571001: done: 0.014906 sec","job_status":"done","queue_duration_s":1.795395,"scheduling_latency_s":1.795412,"redis_calls":13,"redis_duration_s":0.0014399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014906,"completed_at":"2026-09-13T10:49:25.561Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.043061] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64f0fa3ad2c6189ef302a9d","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64f0fa3ad2c6189ef302a9d: start","job_status":"start","queue_duration_s":1.811422,"scheduling_latency_s":1.81144} gitlab # [ 242.055378] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.577Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f64f0fa3ad2c6189ef302a9d","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f64f0fa3ad2c6189ef302a9d: done: 0.015141 sec","job_status":"done","queue_duration_s":1.811422,"scheduling_latency_s":1.81144,"redis_calls":13,"redis_duration_s":0.0015570000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010353,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015141,"completed_at":"2026-09-13T10:49:25.577Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.095342] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e109e04ddbcfef9817194e1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e109e04ddbcfef9817194e1: start","job_status":"start","queue_duration_s":1.827582,"scheduling_latency_s":1.8276} gitlab # [ 242.107899] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.593Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e109e04ddbcfef9817194e1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e109e04ddbcfef9817194e1: done: 0.015118 sec","job_status":"done","queue_duration_s":1.827582,"scheduling_latency_s":1.8276,"redis_calls":13,"redis_duration_s":0.001687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010363,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015118,"completed_at":"2026-09-13T10:49:25.593Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.146982] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b37ea22d3c16d7ba3489fbb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b37ea22d3c16d7ba3489fbb: start","job_status":"start","queue_duration_s":1.843865,"scheduling_latency_s":1.843881} gitlab # [ 242.159438] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.609Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b37ea22d3c16d7ba3489fbb","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b37ea22d3c16d7ba3489fbb: done: 0.014757 sec","job_status":"done","queue_duration_s":1.843865,"scheduling_latency_s":1.843881,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014757,"completed_at":"2026-09-13T10:49:25.609Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.199147] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c60ee0c305bd06a503a459","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c60ee0c305bd06a503a459: start","job_status":"start","queue_duration_s":1.85983,"scheduling_latency_s":1.859847} gitlab # [ 242.211580] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.625Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d0c60ee0c305bd06a503a459","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d0c60ee0c305bd06a503a459: done: 0.014804 sec","job_status":"done","queue_duration_s":1.85983,"scheduling_latency_s":1.859847,"redis_calls":13,"redis_duration_s":0.001467,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010325,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014804,"completed_at":"2026-09-13T10:49:25.625Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.250820] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d88cef608f125735e4663ca4","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d88cef608f125735e4663ca4: start","job_status":"start","queue_duration_s":1.875899,"scheduling_latency_s":1.875918} gitlab # [ 242.263291] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d88cef608f125735e4663ca4","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d88cef608f125735e4663ca4: done: 0.015148 sec","job_status":"done","queue_duration_s":1.875899,"scheduling_latency_s":1.875918,"redis_calls":13,"redis_duration_s":0.001759,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010525,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015148,"completed_at":"2026-09-13T10:49:25.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.303067] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29260b98b0a558476edf7dd3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29260b98b0a558476edf7dd3: start","job_status":"start","queue_duration_s":1.892338,"scheduling_latency_s":1.892355} gitlab # [ 242.315350] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"29260b98b0a558476edf7dd3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-29260b98b0a558476edf7dd3: done: 0.015342 sec","job_status":"done","queue_duration_s":1.892338,"scheduling_latency_s":1.892355,"redis_calls":13,"redis_duration_s":0.001441,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015342,"completed_at":"2026-09-13T10:49:25.658Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.354776] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ae3ff03d455546a789e70f3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ae3ff03d455546a789e70f3: start","job_status":"start","queue_duration_s":1.908972,"scheduling_latency_s":1.908991} gitlab # [ 242.367153] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ae3ff03d455546a789e70f3","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ae3ff03d455546a789e70f3: done: 0.014961 sec","job_status":"done","queue_duration_s":1.908972,"scheduling_latency_s":1.908991,"redis_calls":13,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014961,"completed_at":"2026-09-13T10:49:25.675Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.405958] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f4d19f132b60530b734e3b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f4d19f132b60530b734e3b: start","job_status":"start","queue_duration_s":1.925185,"scheduling_latency_s":1.925204} gitlab # [ 242.418427] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.691Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30f4d19f132b60530b734e3b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30f4d19f132b60530b734e3b: done: 0.014819 sec","job_status":"done","queue_duration_s":1.925185,"scheduling_latency_s":1.925204,"redis_calls":13,"redis_duration_s":0.001428,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010403,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014819,"completed_at":"2026-09-13T10:49:25.691Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.456991] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983a28300ace2c9bea4c3a2b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983a28300ace2c9bea4c3a2b: start","job_status":"start","queue_duration_s":1.940909,"scheduling_latency_s":1.940926} gitlab # [ 242.469157] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"983a28300ace2c9bea4c3a2b","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-983a28300ace2c9bea4c3a2b: done: 0.015372 sec","job_status":"done","queue_duration_s":1.940909,"scheduling_latency_s":1.940926,"redis_calls":13,"redis_duration_s":0.00166,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015372,"completed_at":"2026-09-13T10:49:25.707Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.508556] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3eb7ebba02062de5234e9a1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3eb7ebba02062de5234e9a1: start","job_status":"start","queue_duration_s":1.957347,"scheduling_latency_s":1.957364} gitlab # [ 242.520906] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3eb7ebba02062de5234e9a1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3eb7ebba02062de5234e9a1: done: 0.015067 sec","job_status":"done","queue_duration_s":1.957347,"scheduling_latency_s":1.957364,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015067,"completed_at":"2026-09-13T10:49:25.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.559870] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2a29349af5306a91fd18914","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2a29349af5306a91fd18914: start","job_status":"start","queue_duration_s":1.973438,"scheduling_latency_s":1.973456} gitlab # [ 242.572131] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e2a29349af5306a91fd18914","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e2a29349af5306a91fd18914: done: 0.015423 sec","job_status":"done","queue_duration_s":1.973438,"scheduling_latency_s":1.973456,"redis_calls":13,"redis_duration_s":0.001409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015423,"completed_at":"2026-09-13T10:49:25.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.610500] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"301599abd50ccf616f0b8997","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-301599abd50ccf616f0b8997: start","job_status":"start","queue_duration_s":1.989931,"scheduling_latency_s":1.98995} gitlab # [ 242.622866] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"301599abd50ccf616f0b8997","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-301599abd50ccf616f0b8997: done: 0.015464 sec","job_status":"done","queue_duration_s":1.989931,"scheduling_latency_s":1.98995,"redis_calls":13,"redis_duration_s":0.0014880000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015464,"completed_at":"2026-09-13T10:49:25.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.661283] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9de1be2478994b919910adf","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9de1be2478994b919910adf: start","job_status":"start","queue_duration_s":2.006476,"scheduling_latency_s":2.006494} gitlab # [ 242.673628] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9de1be2478994b919910adf","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9de1be2478994b919910adf: done: 0.014545 sec","job_status":"done","queue_duration_s":2.006476,"scheduling_latency_s":2.006494,"redis_calls":13,"redis_duration_s":0.001571,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-09-13T10:49:25.772Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.712830] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.773Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bfa68afe44d9108651debe1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bfa68afe44d9108651debe1: start","job_status":"start","queue_duration_s":2.022241,"scheduling_latency_s":2.022259} gitlab # [ 242.724723] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1bfa68afe44d9108651debe1","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1bfa68afe44d9108651debe1: done: 0.014928 sec","job_status":"done","queue_duration_s":2.022241,"scheduling_latency_s":2.022259,"redis_calls":13,"redis_duration_s":0.001516,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014928,"completed_at":"2026-09-13T10:49:25.788Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.763646] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb67e32724f655013139ab76","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb67e32724f655013139ab76: start","job_status":"start","queue_duration_s":2.0384,"scheduling_latency_s":2.038418} gitlab # [ 242.775968] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb67e32724f655013139ab76","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb67e32724f655013139ab76: done: 0.01484 sec","job_status":"done","queue_duration_s":2.0384,"scheduling_latency_s":2.038418,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010438,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01484,"completed_at":"2026-09-13T10:49:25.804Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.814080] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.805Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bdf6c4ccbc31ef2ef1ae692","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bdf6c4ccbc31ef2ef1ae692: start","job_status":"start","queue_duration_s":2.054582,"scheduling_latency_s":2.0546} gitlab # [ 242.826451] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5bdf6c4ccbc31ef2ef1ae692","created_at":"2026-09-13T10:49:23.211Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-0788815d65704874","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2d07a60abd99a58e0fdc3283a2a6c2bb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.751Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5bdf6c4ccbc31ef2ef1ae692: done: 0.014887 sec","job_status":"done","queue_duration_s":2.054582,"scheduling_latency_s":2.0546,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014887,"completed_at":"2026-09-13T10:49:25.820Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.864361] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:25.821Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3e79162b201c64dcc38e884d","created_at":"2026-09-13T10:49:23.754Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9e3c4ec48a0bfb349a7a7411ef45f63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.785Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3e79162b201c64dcc38e884d: start","job_status":"start","queue_duration_s":2.036627,"scheduling_latency_s":2.036646} gitlab # [ 242.877789] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.083Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"3e79162b201c64dcc38e884d","created_at":"2026-09-13T10:49:23.754Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9e3c4ec48a0bfb349a7a7411ef45f63","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.785Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-3e79162b201c64dcc38e884d: done: 0.261963 sec","job_status":"done","queue_duration_s":2.036627,"scheduling_latency_s":2.036646,"redis_calls":12,"redis_duration_s":0.0017310000000000001,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000755,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000688,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.052,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.051,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.192613,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00066816,"exclusive_lock_hold_duration_s":0.24118402,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.261963,"completed_at":"2026-09-13T10:49:26.083Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.077068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.922595] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"400c4c7311daa0dd70b3a3f9","created_at":"2026-09-13T10:49:23.822Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45c8ca1da5ee526e227a1b9ad6b644a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.916Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-400c4c7311daa0dd70b3a3f9: start","job_status":"start","queue_duration_s":2.168388,"scheduling_latency_s":2.168408} gitlab # [ 242.936915] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.316Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"400c4c7311daa0dd70b3a3f9","created_at":"2026-09-13T10:49:23.822Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f45c8ca1da5ee526e227a1b9ad6b644a","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.916Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-400c4c7311daa0dd70b3a3f9: done: 0.231068 sec","job_status":"done","queue_duration_s":2.168388,"scheduling_latency_s":2.168408,"redis_calls":12,"redis_duration_s":0.003102,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000302,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.208431,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00044726,"exclusive_lock_hold_duration_s":0.20337148000000002,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.231068,"completed_at":"2026-09-13T10:49:26.316Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013008,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 242.980356] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.319Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"03edf834774c6bfee287eace","created_at":"2026-09-13T10:49:23.934Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4006c72bc193f2165e2391d56b778dfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:49:23.948Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-03edf834774c6bfee287eace: start","job_status":"start","queue_duration_s":2.371151,"scheduling_latency_s":2.37119} gitlab # [ 242.993309] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"03edf834774c6bfee287eace","created_at":"2026-09-13T10:49:23.934Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4006c72bc193f2165e2391d56b778dfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:49:23.948Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-03edf834774c6bfee287eace: done: 0.062823 sec","job_status":"done","queue_duration_s":2.371151,"scheduling_latency_s":2.37119,"redis_calls":10,"redis_duration_s":0.003701,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000755,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002228,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000718,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00075478,"duration_s":0.062823,"completed_at":"2026-09-13T10:49:26.381Z","load_balancing_strategy":"primary","db_duration_s":0.007146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.033704] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.384Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc56ab4ea04c5f6993bd7ea1","created_at":"2026-09-13T10:49:23.963Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3abb55139caa8fd84a0639370764598c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:49:23.975Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc56ab4ea04c5f6993bd7ea1: start","job_status":"start","queue_duration_s":2.409564,"scheduling_latency_s":2.409594} gitlab # [ 243.047037] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.403Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 243.050928] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fc56ab4ea04c5f6993bd7ea1","created_at":"2026-09-13T10:49:23.963Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3abb55139caa8fd84a0639370764598c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:49:23.975Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fc56ab4ea04c5f6993bd7ea1: done: 0.029958 sec","job_status":"done","queue_duration_s":2.409564,"scheduling_latency_s":2.409594,"redis_calls":10,"redis_duration_s":0.002895,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000626,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000574,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00044526,"duration_s":0.029958,"completed_at":"2026-09-13T10:49:26.414Z","load_balancing_strategy":"primary","db_duration_s":0.000807,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.091532] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9b9c5cf7ba6e5bde2abe8b64","created_at":"2026-09-13T10:49:23.981Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f823a44351853aafaeca593d046b8442","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.991Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9b9c5cf7ba6e5bde2abe8b64: start","job_status":"start","queue_duration_s":2.4251,"scheduling_latency_s":2.425125} gitlab # [ 243.106961] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.433Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 243.110465] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.443Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"9b9c5cf7ba6e5bde2abe8b64","created_at":"2026-09-13T10:49:23.981Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f823a44351853aafaeca593d046b8442","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:23.991Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-9b9c5cf7ba6e5bde2abe8b64: done: 0.026722 sec","job_status":"done","queue_duration_s":2.4251,"scheduling_latency_s":2.425125,"redis_calls":10,"redis_duration_s":0.002622,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001569,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003691,"duration_s":0.026722,"completed_at":"2026-09-13T10:49:26.443Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000672,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.152623] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"59233000c76f572d885c7404","created_at":"2026-09-13T10:49:24.004Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"616f7f1ec86163bff05fd959628f8239","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.013Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-59233000c76f572d885c7404: start","job_status":"start","queue_duration_s":2.43217,"scheduling_latency_s":2.432193} gitlab # [ 243.167186] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.484Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"59233000c76f572d885c7404","created_at":"2026-09-13T10:49:24.004Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"616f7f1ec86163bff05fd959628f8239","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.013Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-59233000c76f572d885c7404: done: 0.03915 sec","job_status":"done","queue_duration_s":2.43217,"scheduling_latency_s":2.432193,"redis_calls":13,"redis_duration_s":0.003367,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00053,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001703,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001134,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012099500000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.03915,"completed_at":"2026-09-13T10:49:26.484Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004894,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 243.211038] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.486Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f6c09a5da2f5469ce17b3784","created_at":"2026-09-13T10:49:24.032Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078a5a15a24ba8e573ad71da3d50387e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.047Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f6c09a5da2f5469ce17b3784: start","job_status":"start","queue_duration_s":2.438796,"scheduling_latency_s":2.438814} gitlab # [ 243.224955] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"f6c09a5da2f5469ce17b3784","created_at":"2026-09-13T10:49:24.032Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"078a5a15a24ba8e573ad71da3d50387e","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.047Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-f6c09a5da2f5469ce17b3784: done: 0.057711 sec","job_status":"done","queue_duration_s":2.438796,"scheduling_latency_s":2.438814,"redis_calls":10,"redis_duration_s":0.002888,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000507,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000582,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0008131,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.057711,"completed_at":"2026-09-13T10:49:26.544Z","load_balancing_strategy":"primary","db_duration_s":0.030039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.267608] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2e5a3b5ca489a14cc372173e","created_at":"2026-09-13T10:49:24.076Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46be691cf9f2d026cce6515d9c2f222e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.084Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2e5a3b5ca489a14cc372173e: start","job_status":"start","queue_duration_s":2.46231,"scheduling_latency_s":2.462336} gitlab # [ 243.281150] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"2e5a3b5ca489a14cc372173e","created_at":"2026-09-13T10:49:24.076Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46be691cf9f2d026cce6515d9c2f222e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.084Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-2e5a3b5ca489a14cc372173e: done: 0.037329 sec","job_status":"done","queue_duration_s":2.46231,"scheduling_latency_s":2.462336,"redis_calls":12,"redis_duration_s":0.00367,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000604,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001658,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069044,"exclusive_lock_hold_duration_s":0.0102147,"duration_s":0.037329,"completed_at":"2026-09-13T10:49:26.583Z","load_balancing_strategy":"primary","db_duration_s":0.005416,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.322868] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.585Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"592aa3eca680a132bbc26408","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-592aa3eca680a132bbc26408: start","job_status":"start","queue_duration_s":1.600969,"scheduling_latency_s":1.600993} gitlab # [ 243.334965] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.607Z","message":"No existing merge request to be cleaned up."} gitlab # [ 243.336613] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.624Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"592aa3eca680a132bbc26408","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-592aa3eca680a132bbc26408: done: 0.038519 sec","job_status":"done","queue_duration_s":1.600969,"scheduling_latency_s":1.600993,"redis_calls":13,"redis_duration_s":0.003444,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000523,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001299,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001622,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039922,"duration_s":0.038519,"completed_at":"2026-09-13T10:49:26.624Z","load_balancing_strategy":"primary","db_duration_s":0.005459,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.376495] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.626Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"938453fb15724b714ae37450","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-938453fb15724b714ae37450: start","job_status":"start","queue_duration_s":1.64133,"scheduling_latency_s":1.641351} gitlab # [ 243.388634] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.638Z","message":"No existing merge request to be cleaned up."} gitlab # [ 243.390316] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.649Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"938453fb15724b714ae37450","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-938453fb15724b714ae37450: done: 0.023241 sec","job_status":"done","queue_duration_s":1.64133,"scheduling_latency_s":1.641351,"redis_calls":10,"redis_duration_s":0.0031660000000000004,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002043,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023241,"completed_at":"2026-09-13T10:49:26.649Z","load_balancing_strategy":"primary","db_duration_s":0.002597,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.428457] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.651Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fec4d809d3bab3655229a273","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fec4d809d3bab3655229a273: start","job_status":"start","queue_duration_s":1.666215,"scheduling_latency_s":1.666234} gitlab # [ 243.440487] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.663Z","message":"No existing merge request to be cleaned up."} gitlab # [ 243.442091] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.672Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fec4d809d3bab3655229a273","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fec4d809d3bab3655229a273: done: 0.021407 sec","job_status":"done","queue_duration_s":1.666215,"scheduling_latency_s":1.666234,"redis_calls":10,"redis_duration_s":0.00258,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00104,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00154,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021407,"completed_at":"2026-09-13T10:49:26.672Z","load_balancing_strategy":"primary","db_duration_s":0.002498,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.479858] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.673Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fa77436fa47686acd019d4a5","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fa77436fa47686acd019d4a5: start","job_status":"start","queue_duration_s":1.688982,"scheduling_latency_s":1.689} gitlab # [ 243.492120] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.685Z","message":"No existing merge request to be cleaned up."} gitlab # [ 243.493697] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:49:26.694Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fa77436fa47686acd019d4a5","created_at":"2026-09-13T10:49:24.962Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-37068d1f8d9743dc","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"970a23e221ae071bda315ce1b09c3cf1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:49:24.984Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fa77436fa47686acd019d4a5: done: 0.020665 sec","job_status":"done","queue_duration_s":1.688982,"scheduling_latency_s":1.689,"redis_calls":10,"redis_duration_s":0.0025700000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001162,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020665,"completed_at":"2026-09-13T10:49:26.694Z","load_balancing_strategy":"primary","db_duration_s":0.002609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.069740] setup-gitlab-start[2980]: W, [2026-09-13T10:50:01.943741 #2980] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 279.066375] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:03.939Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0668972a79ee4f1d49bb9a47","created_at":"2026-09-13T10:50:03.919Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:03.936Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0668972a79ee4f1d49bb9a47: start","job_status":"start","queue_duration_s":0.003466,"scheduling_latency_s":0.00353} gitlab # [ 279.666938] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:04.541Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"0668972a79ee4f1d49bb9a47","created_at":"2026-09-13T10:50:03.919Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:03.936Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-0668972a79ee4f1d49bb9a47: done: 0.60149 sec","job_status":"done","queue_duration_s":0.003466,"scheduling_latency_s":0.00353,"redis_calls":60,"redis_duration_s":0.24285800000000002,"redis_read_bytes":108,"redis_write_bytes":40959,"redis_queues_calls":2,"redis_queues_duration_s":0.00183,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37335,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.234604,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.006424,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.257331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.60149,"completed_at":"2026-09-13T10:50:04.540Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.755733] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:04.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"3f5855947920da6fe44d3104","created_at":"2026-09-13T10:50:03.969Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc9abd19591d80a3cbf3b7196fd0b28c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:03.997Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-3f5855947920da6fe44d3104: start","job_status":"start","queue_duration_s":0.569513,"scheduling_latency_s":0.569574} gitlab # [ 280.067850] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:04.941Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivateExpiredDeploymentsCronWorker","jid":"3f5855947920da6fe44d3104","created_at":"2026-09-13T10:50:03.969Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fc9abd19591d80a3cbf3b7196fd0b28c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:787f72254f5c8a3553ab694c3728fa911b46196df537a2b1a46751f692c5ffa6","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:03.997Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivateExpiredDeploymentsCronWorker JID-3f5855947920da6fe44d3104: done: 0.374144 sec","job_status":"done","queue_duration_s":0.569513,"scheduling_latency_s":0.569574,"redis_calls":12,"redis_duration_s":0.021056,"redis_read_bytes":265,"redis_write_bytes":1300,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.007153,"redis_feature_flag_read_bytes":258,"redis_feature_flag_write_bytes":446,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004195,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.084,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.084,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00263548,"extra.pages_deactivate_expired_deployments_cron_worker.deactivate_expired_pages_deployments":{"deactivated_deployments":0,"duration":0.02538062},"duration_s":0.374144,"completed_at":"2026-09-13T10:50:04.941Z","load_balancing_strategy":"primary","db_duration_s":0.094696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.160027] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:04.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a0d1997e9bae0c496c8db265","created_at":"2026-09-13T10:50:04.050Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edba28b3849c5098d64729fb77a8ae1c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.060Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a0d1997e9bae0c496c8db265: start","job_status":"start","queue_duration_s":0.909148,"scheduling_latency_s":0.909189} gitlab # [ 280.382932] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.256Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a0d1997e9bae0c496c8db265","created_at":"2026-09-13T10:50:04.050Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"edba28b3849c5098d64729fb77a8ae1c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.060Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a0d1997e9bae0c496c8db265: done: 0.287173 sec","job_status":"done","queue_duration_s":0.909148,"scheduling_latency_s":0.909189,"redis_calls":9,"redis_duration_s":0.0212,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.009435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.011765,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0022636,"exclusive_lock_hold_duration_s":0.11676406,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.287173,"completed_at":"2026-09-13T10:50:05.256Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.108305,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.486322] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a451efa44c21ae40689ba4b5","created_at":"2026-09-13T10:50:04.082Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3a8f262a7a32250c0d814e127841ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.092Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a451efa44c21ae40689ba4b5: start","job_status":"start","queue_duration_s":1.167697,"scheduling_latency_s":1.167744} gitlab # [ 280.505846] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.296Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a451efa44c21ae40689ba4b5","created_at":"2026-09-13T10:50:04.082Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3a8f262a7a32250c0d814e127841ec8","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.092Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a451efa44c21ae40689ba4b5: done: 0.035999 sec","job_status":"done","queue_duration_s":1.167697,"scheduling_latency_s":1.167744,"redis_calls":9,"redis_duration_s":0.002517,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00086934,"exclusive_lock_hold_duration_s":0.00674386,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.035999,"completed_at":"2026-09-13T10:50:05.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003483,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.549956] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.298Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"59d1b715dbc923dea1572a69","created_at":"2026-09-13T10:50:04.145Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d7b5bdc306f87f3849c0d89c60fdde3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.154Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-59d1b715dbc923dea1572a69: start","job_status":"start","queue_duration_s":1.143417,"scheduling_latency_s":1.143451} gitlab # [ 280.564208] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.312Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 280.567745] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.321Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"59d1b715dbc923dea1572a69","created_at":"2026-09-13T10:50:04.145Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d7b5bdc306f87f3849c0d89c60fdde3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.154Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-59d1b715dbc923dea1572a69: done: 0.023443 sec","job_status":"done","queue_duration_s":1.143417,"scheduling_latency_s":1.143451,"redis_calls":7,"redis_duration_s":0.001671,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023443,"completed_at":"2026-09-13T10:50:05.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.607897] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.323Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3ab80fbd5ddc2f2095ac4233","created_at":"2026-09-13T10:50:04.174Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d2198e1c507d13bf92b6f0022671d9a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.184Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3ab80fbd5ddc2f2095ac4233: start","job_status":"start","queue_duration_s":1.138911,"scheduling_latency_s":1.13894} gitlab # [ 280.622268] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.335Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 280.625475] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3ab80fbd5ddc2f2095ac4233","created_at":"2026-09-13T10:50:04.174Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2d2198e1c507d13bf92b6f0022671d9a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.184Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3ab80fbd5ddc2f2095ac4233: done: 0.018696 sec","job_status":"done","queue_duration_s":1.138911,"scheduling_latency_s":1.13894,"redis_calls":7,"redis_duration_s":0.001397,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000363,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018696,"completed_at":"2026-09-13T10:50:05.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.666079] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"351066ab7b5ff46c6e353c6a","created_at":"2026-09-13T10:50:04.213Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"141f2403599a690d0c8f1978c48ed8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.222Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-351066ab7b5ff46c6e353c6a: start","job_status":"start","queue_duration_s":1.120723,"scheduling_latency_s":1.120748} gitlab # [ 280.679975] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.390Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"351066ab7b5ff46c6e353c6a","created_at":"2026-09-13T10:50:04.213Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"141f2403599a690d0c8f1978c48ed8d7","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.222Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-351066ab7b5ff46c6e353c6a: done: 0.046724 sec","job_status":"done","queue_duration_s":1.120723,"scheduling_latency_s":1.120748,"redis_calls":7,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046724,"completed_at":"2026-09-13T10:50:05.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022953,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.719630] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0354a0c163c0fc277076b0be","created_at":"2026-09-13T10:50:04.245Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c782b95cb12896b732e5cde7628a73c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.254Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0354a0c163c0fc277076b0be: start","job_status":"start","queue_duration_s":1.137671,"scheduling_latency_s":1.137704} gitlab # [ 280.734076] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.406Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 280.737637] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.414Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"0354a0c163c0fc277076b0be","created_at":"2026-09-13T10:50:04.245Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c782b95cb12896b732e5cde7628a73c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.254Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-0354a0c163c0fc277076b0be: done: 0.021945 sec","job_status":"done","queue_duration_s":1.137671,"scheduling_latency_s":1.137704,"redis_calls":7,"redis_duration_s":0.001457,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001058,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021945,"completed_at":"2026-09-13T10:50:05.414Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.777669] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.416Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"092f5624b773b7ccbc8ab87f","created_at":"2026-09-13T10:50:04.274Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d716768497287b36548d34e2e338adc3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.298Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-092f5624b773b7ccbc8ab87f: start","job_status":"start","queue_duration_s":1.118424,"scheduling_latency_s":1.118454} gitlab # [ 280.790836] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.445Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Pages::DeactivatedDeploymentsDeleteCronWorker","jid":"092f5624b773b7ccbc8ab87f","created_at":"2026-09-13T10:50:04.274Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d716768497287b36548d34e2e338adc3","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:02dc87c01bb5b1cecbe6eae0009120537d166f4adbbb83ef540bec12f0f9136a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.298Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Pages::DeactivatedDeploymentsDeleteCronWorker JID-092f5624b773b7ccbc8ab87f: done: 0.028638 sec","job_status":"done","queue_duration_s":1.118424,"scheduling_latency_s":1.118454,"redis_calls":10,"redis_duration_s":0.001702,"redis_read_bytes":9,"redis_write_bytes":1187,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000368,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000364,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029834,"duration_s":0.028638,"completed_at":"2026-09-13T10:50:05.445Z","load_balancing_strategy":"primary","db_duration_s":0.003377,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.831345] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ed430f217c97160b78b6d0c6","created_at":"2026-09-13T10:50:04.325Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f7b1e8e148bf23c9520e5c26003a57f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.336Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ed430f217c97160b78b6d0c6: start","job_status":"start","queue_duration_s":1.110671,"scheduling_latency_s":1.110698} gitlab # [ 280.846974] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ed430f217c97160b78b6d0c6","created_at":"2026-09-13T10:50:04.325Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8f7b1e8e148bf23c9520e5c26003a57f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.336Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ed430f217c97160b78b6d0c6: done: 0.063079 sec","job_status":"done","queue_duration_s":1.110671,"scheduling_latency_s":1.110698,"redis_calls":9,"redis_duration_s":0.0019199999999999998,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070006,"exclusive_lock_hold_duration_s":0.03326146,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.063079,"completed_at":"2026-09-13T10:50:05.509Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.889588] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.512Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"f49a40f120ac43a59fe38f8e","created_at":"2026-09-13T10:50:04.377Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"781b89259e07b13522c277eba12cfa88","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.404Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-f49a40f120ac43a59fe38f8e: start","job_status":"start","queue_duration_s":1.107263,"scheduling_latency_s":1.107294} gitlab # [ 280.902867] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.544Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Projects::InactiveProjectsDeletionCronWorker","jid":"f49a40f120ac43a59fe38f8e","created_at":"2026-09-13T10:50:04.377Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"781b89259e07b13522c277eba12cfa88","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:442251d7c2d07bf110c698169f390fde7a2ca5a69ba3f639f0fb0d3b8866112f","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.404Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::InactiveProjectsDeletionCronWorker JID-f49a40f120ac43a59fe38f8e: done: 0.032651 sec","job_status":"done","queue_duration_s":1.107263,"scheduling_latency_s":1.107294,"redis_calls":10,"redis_duration_s":0.001855,"redis_read_bytes":9,"redis_write_bytes":1179,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000361,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001105,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":610,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":238,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032514,"duration_s":0.032651,"completed_at":"2026-09-13T10:50:05.544Z","load_balancing_strategy":"primary","db_duration_s":0.000714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.943389] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.546Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba68e593a649e4076014042c","created_at":"2026-09-13T10:50:04.432Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e17c5feddd10b1f934d21cb4ac589d06","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.444Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba68e593a649e4076014042c: start","job_status":"start","queue_duration_s":1.101988,"scheduling_latency_s":1.102014} gitlab # [ 280.993386] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.868Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ba68e593a649e4076014042c","created_at":"2026-09-13T10:50:04.432Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e17c5feddd10b1f934d21cb4ac589d06","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.444Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ba68e593a649e4076014042c: done: 0.322124 sec","job_status":"done","queue_duration_s":1.101988,"scheduling_latency_s":1.102014,"redis_calls":562,"redis_duration_s":0.056651,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.055839,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.298647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.322124,"completed_at":"2026-09-13T10:50:05.868Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.039620] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.870Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c12132ea9fe529e067fe9f02","created_at":"2026-09-13T10:50:04.473Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3029ff39f7d0f7fb005c1aedacd3e1c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.484Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c12132ea9fe529e067fe9f02: start","job_status":"start","queue_duration_s":1.385685,"scheduling_latency_s":1.385715} gitlab # [ 281.055620] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c12132ea9fe529e067fe9f02","created_at":"2026-09-13T10:50:04.473Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c3029ff39f7d0f7fb005c1aedacd3e1c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.484Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c12132ea9fe529e067fe9f02: done: 0.015698 sec","job_status":"done","queue_duration_s":1.385685,"scheduling_latency_s":1.385715,"redis_calls":8,"redis_duration_s":0.0011510000000000001,"redis_read_bytes":133,"redis_write_bytes":846,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000132,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000277,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009591,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015698,"completed_at":"2026-09-13T10:50:05.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.098417] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d1c05e29f5e7abef1aeafab","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d1c05e29f5e7abef1aeafab: start","job_status":"start","queue_duration_s":1.391449,"scheduling_latency_s":1.391472} gitlab # [ 281.110801] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0d1c05e29f5e7abef1aeafab","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0d1c05e29f5e7abef1aeafab: done: 0.017574 sec","job_status":"done","queue_duration_s":1.391449,"scheduling_latency_s":1.391472,"redis_calls":13,"redis_duration_s":0.001823,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001197,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011942,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017574,"completed_at":"2026-09-13T10:50:05.905Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.149079] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82d18f099705d608ad5c930e","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82d18f099705d608ad5c930e: start","job_status":"start","queue_duration_s":1.410401,"scheduling_latency_s":1.410422} gitlab # [ 281.161340] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82d18f099705d608ad5c930e","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82d18f099705d608ad5c930e: done: 0.016015 sec","job_status":"done","queue_duration_s":1.410401,"scheduling_latency_s":1.410422,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011136,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016015,"completed_at":"2026-09-13T10:50:05.922Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.199893] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"425b4e3c4e0b56e997720b20","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-425b4e3c4e0b56e997720b20: start","job_status":"start","queue_duration_s":1.4277,"scheduling_latency_s":1.427717} gitlab # [ 281.212117] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"425b4e3c4e0b56e997720b20","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-425b4e3c4e0b56e997720b20: done: 0.014926 sec","job_status":"done","queue_duration_s":1.4277,"scheduling_latency_s":1.427717,"redis_calls":13,"redis_duration_s":0.0014130000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014926,"completed_at":"2026-09-13T10:50:05.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.251762] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b521df406cb95ab33dd02431","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b521df406cb95ab33dd02431: start","job_status":"start","queue_duration_s":1.444517,"scheduling_latency_s":1.444536} gitlab # [ 281.264514] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b521df406cb95ab33dd02431","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b521df406cb95ab33dd02431: done: 0.014848 sec","job_status":"done","queue_duration_s":1.444517,"scheduling_latency_s":1.444536,"redis_calls":13,"redis_duration_s":0.0014420000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014848,"completed_at":"2026-09-13T10:50:05.955Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.302747] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90c2a460ed71758a1323bf71","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90c2a460ed71758a1323bf71: start","job_status":"start","queue_duration_s":1.460436,"scheduling_latency_s":1.460453} gitlab # [ 281.315022] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90c2a460ed71758a1323bf71","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90c2a460ed71758a1323bf71: done: 0.015005 sec","job_status":"done","queue_duration_s":1.460436,"scheduling_latency_s":1.460453,"redis_calls":13,"redis_duration_s":0.001485,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015005,"completed_at":"2026-09-13T10:50:05.971Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.353410] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce4490d5f4f67c8665fb4c28","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce4490d5f4f67c8665fb4c28: start","job_status":"start","queue_duration_s":1.476491,"scheduling_latency_s":1.476509} gitlab # [ 281.365340] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce4490d5f4f67c8665fb4c28","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce4490d5f4f67c8665fb4c28: done: 0.015001 sec","job_status":"done","queue_duration_s":1.476491,"scheduling_latency_s":1.476509,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015001,"completed_at":"2026-09-13T10:50:05.987Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.405651] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:05.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8e500803278d89bcde6a8a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8e500803278d89bcde6a8a: start","job_status":"start","queue_duration_s":1.493262,"scheduling_latency_s":1.493279} gitlab # [ 281.417794] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a8e500803278d89bcde6a8a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a8e500803278d89bcde6a8a: done: 0.015525 sec","job_status":"done","queue_duration_s":1.493262,"scheduling_latency_s":1.493279,"redis_calls":13,"redis_duration_s":0.0015270000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015525,"completed_at":"2026-09-13T10:50:06.004Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.457192] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55ad33b55b683899402d9f37","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55ad33b55b683899402d9f37: start","job_status":"start","queue_duration_s":1.509846,"scheduling_latency_s":1.509863} gitlab # [ 281.469793] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"55ad33b55b683899402d9f37","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-55ad33b55b683899402d9f37: done: 0.014778 sec","job_status":"done","queue_duration_s":1.509846,"scheduling_latency_s":1.509863,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-09-13T10:50:06.020Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.508517] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.021Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e12620f0675fa26c3c3ec247","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e12620f0675fa26c3c3ec247: start","job_status":"start","queue_duration_s":1.525702,"scheduling_latency_s":1.525719} gitlab # [ 281.520733] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e12620f0675fa26c3c3ec247","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e12620f0675fa26c3c3ec247: done: 0.01495 sec","job_status":"done","queue_duration_s":1.525702,"scheduling_latency_s":1.525719,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01495,"completed_at":"2026-09-13T10:50:06.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.559362] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.038Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd8e990dcafa1defe8d806cd","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd8e990dcafa1defe8d806cd: start","job_status":"start","queue_duration_s":1.542544,"scheduling_latency_s":1.542562} gitlab # [ 281.571607] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd8e990dcafa1defe8d806cd","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd8e990dcafa1defe8d806cd: done: 0.014704 sec","job_status":"done","queue_duration_s":1.542544,"scheduling_latency_s":1.542562,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014704,"completed_at":"2026-09-13T10:50:06.053Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.610168] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.054Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f855a54e4ea213404ef03cca","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f855a54e4ea213404ef03cca: start","job_status":"start","queue_duration_s":1.558317,"scheduling_latency_s":1.558334} gitlab # [ 281.622317] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f855a54e4ea213404ef03cca","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f855a54e4ea213404ef03cca: done: 0.014303 sec","job_status":"done","queue_duration_s":1.558317,"scheduling_latency_s":1.558334,"redis_calls":13,"redis_duration_s":0.001692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001178,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009976,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014303,"completed_at":"2026-09-13T10:50:06.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.661564] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"700c6c05a7f4b3d30d2564db","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-700c6c05a7f4b3d30d2564db: start","job_status":"start","queue_duration_s":1.573763,"scheduling_latency_s":1.57378} gitlab # [ 281.673834] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"700c6c05a7f4b3d30d2564db","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-700c6c05a7f4b3d30d2564db: done: 0.014517 sec","job_status":"done","queue_duration_s":1.573763,"scheduling_latency_s":1.57378,"redis_calls":13,"redis_duration_s":0.001475,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014517,"completed_at":"2026-09-13T10:50:06.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.712124] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f7ff36249af3d0990dce078","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f7ff36249af3d0990dce078: start","job_status":"start","queue_duration_s":1.589337,"scheduling_latency_s":1.589355} gitlab # [ 281.724134] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0f7ff36249af3d0990dce078","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0f7ff36249af3d0990dce078: done: 0.014531 sec","job_status":"done","queue_duration_s":1.589337,"scheduling_latency_s":1.589355,"redis_calls":13,"redis_duration_s":0.0016380000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001108,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009685,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014531,"completed_at":"2026-09-13T10:50:06.099Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.762501] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"569e5f4bb210bb7cb467d107","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-569e5f4bb210bb7cb467d107: start","job_status":"start","queue_duration_s":1.605104,"scheduling_latency_s":1.605122} gitlab # [ 281.775043] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"569e5f4bb210bb7cb467d107","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-569e5f4bb210bb7cb467d107: done: 0.014347 sec","job_status":"done","queue_duration_s":1.605104,"scheduling_latency_s":1.605122,"redis_calls":13,"redis_duration_s":0.001443,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009917,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014347,"completed_at":"2026-09-13T10:50:06.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.813544] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c488525737d2b6ddabbb2425","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c488525737d2b6ddabbb2425: start","job_status":"start","queue_duration_s":1.620557,"scheduling_latency_s":1.620574} gitlab # [ 281.825799] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c488525737d2b6ddabbb2425","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c488525737d2b6ddabbb2425: done: 0.014241 sec","job_status":"done","queue_duration_s":1.620557,"scheduling_latency_s":1.620574,"redis_calls":13,"redis_duration_s":0.0015800000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014241,"completed_at":"2026-09-13T10:50:06.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.865534] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"423dc07579870bc684846f47","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-423dc07579870bc684846f47: start","job_status":"start","queue_duration_s":1.63594,"scheduling_latency_s":1.635957} gitlab # [ 281.877596] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.146Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"423dc07579870bc684846f47","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-423dc07579870bc684846f47: done: 0.014274 sec","job_status":"done","queue_duration_s":1.63594,"scheduling_latency_s":1.635957,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014274,"completed_at":"2026-09-13T10:50:06.146Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.916154] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3052ed8f8a4c8ad6fbced6a2","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3052ed8f8a4c8ad6fbced6a2: start","job_status":"start","queue_duration_s":1.651494,"scheduling_latency_s":1.65151} gitlab # [ 281.928396] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.161Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3052ed8f8a4c8ad6fbced6a2","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3052ed8f8a4c8ad6fbced6a2: done: 0.014275 sec","job_status":"done","queue_duration_s":1.651494,"scheduling_latency_s":1.65151,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014275,"completed_at":"2026-09-13T10:50:06.161Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 281.966727] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3062e47c8e8cea2824e8fb","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3062e47c8e8cea2824e8fb: start","job_status":"start","queue_duration_s":1.666903,"scheduling_latency_s":1.66692} gitlab # [ 281.979141] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.177Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df3062e47c8e8cea2824e8fb","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df3062e47c8e8cea2824e8fb: done: 0.014345 sec","job_status":"done","queue_duration_s":1.666903,"scheduling_latency_s":1.66692,"redis_calls":13,"redis_duration_s":0.001477,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009955,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014345,"completed_at":"2026-09-13T10:50:06.177Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.017810] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8261e4fe907ee8c3381505fe","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8261e4fe907ee8c3381505fe: start","job_status":"start","queue_duration_s":1.682341,"scheduling_latency_s":1.682358} gitlab # [ 282.030207] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8261e4fe907ee8c3381505fe","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8261e4fe907ee8c3381505fe: done: 0.014906 sec","job_status":"done","queue_duration_s":1.682341,"scheduling_latency_s":1.682358,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014906,"completed_at":"2026-09-13T10:50:06.193Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.070129] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.194Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e74da0592b65fd84bcc1017","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e74da0592b65fd84bcc1017: start","job_status":"start","queue_duration_s":1.698296,"scheduling_latency_s":1.698313} gitlab # [ 282.082524] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.209Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2e74da0592b65fd84bcc1017","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2e74da0592b65fd84bcc1017: done: 0.015111 sec","job_status":"done","queue_duration_s":1.698296,"scheduling_latency_s":1.698313,"redis_calls":13,"redis_duration_s":0.001581,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001012,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015111,"completed_at":"2026-09-13T10:50:06.209Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.121503] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.210Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87fa9c5be0630ab5b0cdb18d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87fa9c5be0630ab5b0cdb18d: start","job_status":"start","queue_duration_s":1.714526,"scheduling_latency_s":1.714544} gitlab # [ 282.133728] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.224Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87fa9c5be0630ab5b0cdb18d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87fa9c5be0630ab5b0cdb18d: done: 0.014264 sec","job_status":"done","queue_duration_s":1.714526,"scheduling_latency_s":1.714544,"redis_calls":13,"redis_duration_s":0.0014329999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009946,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014264,"completed_at":"2026-09-13T10:50:06.224Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.172350] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.226Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a71ed4ff93b927f2e4cfda","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a71ed4ff93b927f2e4cfda: start","job_status":"start","queue_duration_s":1.729867,"scheduling_latency_s":1.729884} gitlab # [ 282.184454] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e8a71ed4ff93b927f2e4cfda","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e8a71ed4ff93b927f2e4cfda: done: 0.014293 sec","job_status":"done","queue_duration_s":1.729867,"scheduling_latency_s":1.729884,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009938,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014293,"completed_at":"2026-09-13T10:50:06.240Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.222615] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.241Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573cb92cae1c8207b8548b2d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573cb92cae1c8207b8548b2d: start","job_status":"start","queue_duration_s":1.745315,"scheduling_latency_s":1.745333} gitlab # [ 282.234835] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"573cb92cae1c8207b8548b2d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-573cb92cae1c8207b8548b2d: done: 0.014541 sec","job_status":"done","queue_duration_s":1.745315,"scheduling_latency_s":1.745333,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014541,"completed_at":"2026-09-13T10:50:06.255Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.273969] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d80414ec8b3d7af8f3318e93","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d80414ec8b3d7af8f3318e93: start","job_status":"start","queue_duration_s":1.761138,"scheduling_latency_s":1.761156} gitlab # [ 282.285963] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d80414ec8b3d7af8f3318e93","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d80414ec8b3d7af8f3318e93: done: 0.014251 sec","job_status":"done","queue_duration_s":1.761138,"scheduling_latency_s":1.761156,"redis_calls":13,"redis_duration_s":0.0014030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014251,"completed_at":"2026-09-13T10:50:06.271Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.324450] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.272Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"795b2b9271458ab604992be3","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-795b2b9271458ab604992be3: start","job_status":"start","queue_duration_s":1.776483,"scheduling_latency_s":1.7765} gitlab # [ 282.336587] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"795b2b9271458ab604992be3","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-795b2b9271458ab604992be3: done: 0.014295 sec","job_status":"done","queue_duration_s":1.776483,"scheduling_latency_s":1.7765,"redis_calls":13,"redis_duration_s":0.0014169999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014295,"completed_at":"2026-09-13T10:50:06.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.375305] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.288Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee6d469bd508a0c8ac9f1c8d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee6d469bd508a0c8ac9f1c8d: start","job_status":"start","queue_duration_s":1.7919,"scheduling_latency_s":1.791917} gitlab # [ 282.387446] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee6d469bd508a0c8ac9f1c8d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee6d469bd508a0c8ac9f1c8d: done: 0.015477 sec","job_status":"done","queue_duration_s":1.7919,"scheduling_latency_s":1.791917,"redis_calls":13,"redis_duration_s":0.001732,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015477,"completed_at":"2026-09-13T10:50:06.303Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.426044] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b0845258de51c35be932076","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b0845258de51c35be932076: start","job_status":"start","queue_duration_s":1.808453,"scheduling_latency_s":1.80847} gitlab # [ 282.438087] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b0845258de51c35be932076","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b0845258de51c35be932076: done: 0.014544 sec","job_status":"done","queue_duration_s":1.808453,"scheduling_latency_s":1.80847,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010211,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014544,"completed_at":"2026-09-13T10:50:06.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.477519] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98bd724822e8e2e0c863c4a5","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98bd724822e8e2e0c863c4a5: start","job_status":"start","queue_duration_s":1.824102,"scheduling_latency_s":1.824119} gitlab # [ 282.489536] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"98bd724822e8e2e0c863c4a5","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-98bd724822e8e2e0c863c4a5: done: 0.022736 sec","job_status":"done","queue_duration_s":1.824102,"scheduling_latency_s":1.824119,"redis_calls":13,"redis_duration_s":0.001505,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022736,"completed_at":"2026-09-13T10:50:06.342Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.527860] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6393b339b2bf7f0bb5d378a1","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6393b339b2bf7f0bb5d378a1: start","job_status":"start","queue_duration_s":1.847954,"scheduling_latency_s":1.847971} gitlab # [ 282.540351] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6393b339b2bf7f0bb5d378a1","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6393b339b2bf7f0bb5d378a1: done: 0.014473 sec","job_status":"done","queue_duration_s":1.847954,"scheduling_latency_s":1.847971,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014473,"completed_at":"2026-09-13T10:50:06.358Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.578651] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"debc8675277bb681cac6c121","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-debc8675277bb681cac6c121: start","job_status":"start","queue_duration_s":1.86356,"scheduling_latency_s":1.863577} gitlab # [ 282.590938] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"debc8675277bb681cac6c121","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-debc8675277bb681cac6c121: done: 0.014531 sec","job_status":"done","queue_duration_s":1.86356,"scheduling_latency_s":1.863577,"redis_calls":13,"redis_duration_s":0.001422,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00089,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014531,"completed_at":"2026-09-13T10:50:06.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.629072] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.375Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c015b0082bb1b14f280f93d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c015b0082bb1b14f280f93d: start","job_status":"start","queue_duration_s":1.879716,"scheduling_latency_s":1.879734} gitlab # [ 282.641235] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.390Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c015b0082bb1b14f280f93d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c015b0082bb1b14f280f93d: done: 0.014719 sec","job_status":"done","queue_duration_s":1.879716,"scheduling_latency_s":1.879734,"redis_calls":13,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014719,"completed_at":"2026-09-13T10:50:06.390Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.680898] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.391Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12645e5f4997170abd10bbf4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12645e5f4997170abd10bbf4: start","job_status":"start","queue_duration_s":1.895547,"scheduling_latency_s":1.895564} gitlab # [ 282.692978] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12645e5f4997170abd10bbf4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12645e5f4997170abd10bbf4: done: 0.014778 sec","job_status":"done","queue_duration_s":1.895547,"scheduling_latency_s":1.895564,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010302,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014778,"completed_at":"2026-09-13T10:50:06.406Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.731433] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e88bf1cf30d58331a00a72","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e88bf1cf30d58331a00a72: start","job_status":"start","queue_duration_s":1.911433,"scheduling_latency_s":1.911451} gitlab # [ 282.743630] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0e88bf1cf30d58331a00a72","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0e88bf1cf30d58331a00a72: done: 0.015047 sec","job_status":"done","queue_duration_s":1.911433,"scheduling_latency_s":1.911451,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015047,"completed_at":"2026-09-13T10:50:06.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.782252] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750c742b48e4a1b5a65ad5ca","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750c742b48e4a1b5a65ad5ca: start","job_status":"start","queue_duration_s":1.927984,"scheduling_latency_s":1.928002} gitlab # [ 282.794376] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"750c742b48e4a1b5a65ad5ca","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-750c742b48e4a1b5a65ad5ca: done: 0.014449 sec","job_status":"done","queue_duration_s":1.927984,"scheduling_latency_s":1.928002,"redis_calls":13,"redis_duration_s":0.00141,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014449,"completed_at":"2026-09-13T10:50:06.438Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.832558] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201d80bffa8ece8318a01300","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201d80bffa8ece8318a01300: start","job_status":"start","queue_duration_s":1.943538,"scheduling_latency_s":1.943555} gitlab # [ 282.844669] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"201d80bffa8ece8318a01300","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-201d80bffa8ece8318a01300: done: 0.014497 sec","job_status":"done","queue_duration_s":1.943538,"scheduling_latency_s":1.943555,"redis_calls":13,"redis_duration_s":0.001413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000905,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010197,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014497,"completed_at":"2026-09-13T10:50:06.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.883849] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9cb1de7d3843502155f193b","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9cb1de7d3843502155f193b: start","job_status":"start","queue_duration_s":1.9592,"scheduling_latency_s":1.959218} gitlab # [ 282.896193] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9cb1de7d3843502155f193b","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9cb1de7d3843502155f193b: done: 0.014587 sec","job_status":"done","queue_duration_s":1.9592,"scheduling_latency_s":1.959218,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014587,"completed_at":"2026-09-13T10:50:06.469Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.934230] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001d7bd9ed188e2ff4ff0568","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001d7bd9ed188e2ff4ff0568: start","job_status":"start","queue_duration_s":1.974866,"scheduling_latency_s":1.974883} gitlab # [ 282.946437] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"001d7bd9ed188e2ff4ff0568","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-001d7bd9ed188e2ff4ff0568: done: 0.014405 sec","job_status":"done","queue_duration_s":1.974866,"scheduling_latency_s":1.974883,"redis_calls":13,"redis_duration_s":0.001415,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014405,"completed_at":"2026-09-13T10:50:06.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 282.984451] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8255e1e388e072d09d6e0aac","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8255e1e388e072d09d6e0aac: start","job_status":"start","queue_duration_s":1.990727,"scheduling_latency_s":1.990761} gitlab # [ 282.996373] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.501Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8255e1e388e072d09d6e0aac","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8255e1e388e072d09d6e0aac: done: 0.014983 sec","job_status":"done","queue_duration_s":1.990727,"scheduling_latency_s":1.990761,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000751,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010389,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014983,"completed_at":"2026-09-13T10:50:06.501Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.034391] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1082e40e4a7b5a39b30973a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1082e40e4a7b5a39b30973a: start","job_status":"start","queue_duration_s":2.006828,"scheduling_latency_s":2.006846} gitlab # [ 283.046465] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.517Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1082e40e4a7b5a39b30973a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1082e40e4a7b5a39b30973a: done: 0.014466 sec","job_status":"done","queue_duration_s":2.006828,"scheduling_latency_s":2.006846,"redis_calls":13,"redis_duration_s":0.0014360000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014466,"completed_at":"2026-09-13T10:50:06.517Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.085057] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.518Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ec038d83df72c7e09a6215a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ec038d83df72c7e09a6215a: start","job_status":"start","queue_duration_s":2.022331,"scheduling_latency_s":2.022349} gitlab # [ 283.098094] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.533Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ec038d83df72c7e09a6215a","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ec038d83df72c7e09a6215a: done: 0.01476 sec","job_status":"done","queue_duration_s":2.022331,"scheduling_latency_s":2.022349,"redis_calls":13,"redis_duration_s":0.001445,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01476,"completed_at":"2026-09-13T10:50:06.533Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.136797] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"749cc415299eccdf4dcfc83c","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-749cc415299eccdf4dcfc83c: start","job_status":"start","queue_duration_s":2.03819,"scheduling_latency_s":2.038207} gitlab # [ 283.149041] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.549Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"749cc415299eccdf4dcfc83c","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-749cc415299eccdf4dcfc83c: done: 0.015138 sec","job_status":"done","queue_duration_s":2.03819,"scheduling_latency_s":2.038207,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015138,"completed_at":"2026-09-13T10:50:06.549Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.187166] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.550Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73c6f2959c80114f9d2174f4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73c6f2959c80114f9d2174f4: start","job_status":"start","queue_duration_s":2.054381,"scheduling_latency_s":2.054399} gitlab # [ 283.199326] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.564Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73c6f2959c80114f9d2174f4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73c6f2959c80114f9d2174f4: done: 0.014294 sec","job_status":"done","queue_duration_s":2.054381,"scheduling_latency_s":2.054399,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014294,"completed_at":"2026-09-13T10:50:06.564Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.237599] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595cbb58aae12d5292bd779f","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595cbb58aae12d5292bd779f: start","job_status":"start","queue_duration_s":2.070145,"scheduling_latency_s":2.070164} gitlab # [ 283.249756] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.581Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"595cbb58aae12d5292bd779f","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-595cbb58aae12d5292bd779f: done: 0.015365 sec","job_status":"done","queue_duration_s":2.070145,"scheduling_latency_s":2.070164,"redis_calls":13,"redis_duration_s":0.001605,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001054,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010639,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015365,"completed_at":"2026-09-13T10:50:06.581Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.288198] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"281af7e8d4855cdcc5b54847","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-281af7e8d4855cdcc5b54847: start","job_status":"start","queue_duration_s":2.086597,"scheduling_latency_s":2.086615} gitlab # [ 283.301333] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.597Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"281af7e8d4855cdcc5b54847","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-281af7e8d4855cdcc5b54847: done: 0.014661 sec","job_status":"done","queue_duration_s":2.086597,"scheduling_latency_s":2.086615,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014661,"completed_at":"2026-09-13T10:50:06.597Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.339743] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e0dd786f3e8a71ccbaa55d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e0dd786f3e8a71ccbaa55d: start","job_status":"start","queue_duration_s":2.102327,"scheduling_latency_s":2.102344} gitlab # [ 283.352036] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8e0dd786f3e8a71ccbaa55d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8e0dd786f3e8a71ccbaa55d: done: 0.014939 sec","job_status":"done","queue_duration_s":2.102327,"scheduling_latency_s":2.102344,"redis_calls":13,"redis_duration_s":0.0014520000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014939,"completed_at":"2026-09-13T10:50:06.613Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.390686] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4108250a746e0f2bfc2542d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4108250a746e0f2bfc2542d: start","job_status":"start","queue_duration_s":2.119702,"scheduling_latency_s":2.119721} gitlab # [ 283.402846] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4108250a746e0f2bfc2542d","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4108250a746e0f2bfc2542d: done: 0.01501 sec","job_status":"done","queue_duration_s":2.119702,"scheduling_latency_s":2.119721,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01501,"completed_at":"2026-09-13T10:50:06.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.440907] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8600da52faee66691444fda9","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8600da52faee66691444fda9: start","job_status":"start","queue_duration_s":2.135815,"scheduling_latency_s":2.135832} gitlab # [ 283.452824] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8600da52faee66691444fda9","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8600da52faee66691444fda9: done: 0.015148 sec","job_status":"done","queue_duration_s":2.135815,"scheduling_latency_s":2.135832,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015148,"completed_at":"2026-09-13T10:50:06.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.490856] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce1c7d90e0e5f357a9b9d276","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce1c7d90e0e5f357a9b9d276: start","job_status":"start","queue_duration_s":2.152053,"scheduling_latency_s":2.152071} gitlab # [ 283.504053] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce1c7d90e0e5f357a9b9d276","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce1c7d90e0e5f357a9b9d276: done: 0.014703 sec","job_status":"done","queue_duration_s":2.152053,"scheduling_latency_s":2.152071,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014703,"completed_at":"2026-09-13T10:50:06.662Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.541997] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"928aefb1afabcd349a07b5f4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-928aefb1afabcd349a07b5f4: start","job_status":"start","queue_duration_s":2.167858,"scheduling_latency_s":2.167876} gitlab # [ 283.554187] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"928aefb1afabcd349a07b5f4","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-928aefb1afabcd349a07b5f4: done: 0.014765 sec","job_status":"done","queue_duration_s":2.167858,"scheduling_latency_s":2.167876,"redis_calls":13,"redis_duration_s":0.001424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014765,"completed_at":"2026-09-13T10:50:06.678Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.592270] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9800086dc2eae4b3745b61af","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9800086dc2eae4b3745b61af: start","job_status":"start","queue_duration_s":2.183815,"scheduling_latency_s":2.183833} gitlab # [ 283.604382] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9800086dc2eae4b3745b61af","created_at":"2026-09-13T10:50:03.995Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-9b2f10c5193140b2","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"ad73772d187ab494f3edbf7b54788cbb","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.496Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9800086dc2eae4b3745b61af: done: 0.015578 sec","job_status":"done","queue_duration_s":2.183815,"scheduling_latency_s":2.183833,"redis_calls":13,"redis_duration_s":0.0014,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000509,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015578,"completed_at":"2026-09-13T10:50:06.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.643359] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.696Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"80f1309584b06bd04d80d2d9","created_at":"2026-09-13T10:50:04.511Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae38d527c939f19b2bad92754d162490","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.520Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-80f1309584b06bd04d80d2d9: start","job_status":"start","queue_duration_s":2.176065,"scheduling_latency_s":2.176082} gitlab # [ 283.657687] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.710Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"80f1309584b06bd04d80d2d9","created_at":"2026-09-13T10:50:04.511Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ae38d527c939f19b2bad92754d162490","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.520Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-80f1309584b06bd04d80d2d9: done: 0.013645 sec","job_status":"done","queue_duration_s":2.176065,"scheduling_latency_s":2.176082,"redis_calls":7,"redis_duration_s":0.0012070000000000002,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000711,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000496,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013645,"completed_at":"2026-09-13T10:50:06.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.697931] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6b62519bef0a026e737a71cb","created_at":"2026-09-13T10:50:04.543Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78eb3487d3f4378525c03bf96fb93686","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.550Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6b62519bef0a026e737a71cb: start","job_status":"start","queue_duration_s":2.160715,"scheduling_latency_s":2.160732} gitlab # [ 283.712661] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6b62519bef0a026e737a71cb","created_at":"2026-09-13T10:50:04.543Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"78eb3487d3f4378525c03bf96fb93686","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.550Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6b62519bef0a026e737a71cb: done: 0.01671 sec","job_status":"done","queue_duration_s":2.160715,"scheduling_latency_s":2.160732,"redis_calls":7,"redis_duration_s":0.0009989999999999999,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01671,"completed_at":"2026-09-13T10:50:06.728Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.752420] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.729Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bf8299a52b6ff02c68dcc466","created_at":"2026-09-13T10:50:04.565Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc60ae7119aa79af9b4095e296d7960c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:50:04.573Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bf8299a52b6ff02c68dcc466: start","job_status":"start","queue_duration_s":2.155944,"scheduling_latency_s":2.155962} gitlab # [ 283.765410] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.737Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 283.768608] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.742Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"bf8299a52b6ff02c68dcc466","created_at":"2026-09-13T10:50:04.565Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cc60ae7119aa79af9b4095e296d7960c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:50:04.573Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-bf8299a52b6ff02c68dcc466: done: 0.013039 sec","job_status":"done","queue_duration_s":2.155944,"scheduling_latency_s":2.155962,"redis_calls":7,"redis_duration_s":0.000957,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000262,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013039,"completed_at":"2026-09-13T10:50:06.742Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.807713] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"dd559e192ed1edd2a539d9d3","created_at":"2026-09-13T10:50:04.592Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db5a14a59be7316a7d36fad14280a552","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.604Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-dd559e192ed1edd2a539d9d3: start","job_status":"start","queue_duration_s":2.138337,"scheduling_latency_s":2.138356} gitlab # [ 283.820807] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.809Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Releases::PublishEventWorker","jid":"dd559e192ed1edd2a539d9d3","created_at":"2026-09-13T10:50:04.592Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db5a14a59be7316a7d36fad14280a552","meta.root_caller_id":"Cronjob","meta.feature_category":"release_orchestration","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9f5dbd6b3e0f8a09b95f6e4bc38c93c1137695f37704af6aead7c9933a73fb55","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.604Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Releases::PublishEventWorker JID-dd559e192ed1edd2a539d9d3: done: 0.065996 sec","job_status":"done","queue_duration_s":2.138337,"scheduling_latency_s":2.138356,"redis_calls":10,"redis_duration_s":0.001395,"redis_read_bytes":9,"redis_write_bytes":1039,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000791,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027462,"duration_s":0.065996,"completed_at":"2026-09-13T10:50:06.809Z","load_balancing_strategy":"primary","db_duration_s":0.031187,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.861644] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.810Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6771b6b3ebdc2c4e57d0c4fa","created_at":"2026-09-13T10:50:04.614Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"003ebac33af863e6c3b06f22cff2cc50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.626Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6771b6b3ebdc2c4e57d0c4fa: start","job_status":"start","queue_duration_s":2.183651,"scheduling_latency_s":2.18367} gitlab # [ 283.876043] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"6771b6b3ebdc2c4e57d0c4fa","created_at":"2026-09-13T10:50:04.614Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"003ebac33af863e6c3b06f22cff2cc50","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.626Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-6771b6b3ebdc2c4e57d0c4fa: done: 0.025045 sec","job_status":"done","queue_duration_s":2.183651,"scheduling_latency_s":2.18367,"redis_calls":12,"redis_duration_s":0.001675,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000277,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000653,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005009,"exclusive_lock_hold_duration_s":0.00626338,"duration_s":0.025045,"completed_at":"2026-09-13T10:50:06.835Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.919196] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.837Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6d6438fc0fe90d38318015b5","created_at":"2026-09-13T10:50:04.644Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd495c98003600b00930b615ffb5600","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.655Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6d6438fc0fe90d38318015b5: start","job_status":"start","queue_duration_s":2.181516,"scheduling_latency_s":2.181535} gitlab # [ 283.933778] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.865Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"6d6438fc0fe90d38318015b5","created_at":"2026-09-13T10:50:04.644Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ccd495c98003600b00930b615ffb5600","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.655Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-6d6438fc0fe90d38318015b5: done: 0.027983 sec","job_status":"done","queue_duration_s":2.181516,"scheduling_latency_s":2.181535,"redis_calls":10,"redis_duration_s":0.001336,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000797,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023764,"duration_s":0.027983,"completed_at":"2026-09-13T10:50:06.865Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004949,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 283.975769] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.866Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"7f7f430fc69c7d9025e4094e","created_at":"2026-09-13T10:50:04.679Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b62b26468046a1072da9a277cd605bee","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.740Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-7f7f430fc69c7d9025e4094e: start","job_status":"start","queue_duration_s":2.125402,"scheduling_latency_s":2.12542} gitlab # [ 283.990605] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Packages::Cleanup::DeleteOrphanedDependenciesWorker","jid":"7f7f430fc69c7d9025e4094e","created_at":"2026-09-13T10:50:04.679Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b62b26468046a1072da9a277cd605bee","meta.root_caller_id":"Cronjob","meta.feature_category":"package_registry","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ae1976bd21c961bb04f5dedb6e2b63c4b50acd9c52a47e7573ade27e3c65ce36","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.740Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Packages::Cleanup::DeleteOrphanedDependenciesWorker JID-7f7f430fc69c7d9025e4094e: done: 0.034584 sec","job_status":"done","queue_duration_s":2.125402,"scheduling_latency_s":2.12542,"redis_calls":13,"redis_duration_s":0.001796,"redis_read_bytes":10,"redis_write_bytes":1343,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000338,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":345,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000698,"redis_shared_state_read_bytes":4,"redis_shared_state_write_bytes":368,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002381,"extra.packages_cleanup_delete_orphaned_dependencies_worker.last_processed_packages_dependency_id":0,"extra.packages_cleanup_delete_orphaned_dependencies_worker.deleted_rows_count":0,"duration_s":0.034584,"completed_at":"2026-09-13T10:50:06.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.034807] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"add6ad88168568b259f7df40","created_at":"2026-09-13T10:50:04.753Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.760Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-add6ad88168568b259f7df40: start","job_status":"start","queue_duration_s":2.142122,"scheduling_latency_s":2.142144} gitlab # [ 284.048427] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"add6ad88168568b259f7df40","created_at":"2026-09-13T10:50:04.753Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.760Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-add6ad88168568b259f7df40: done: 0.026813 sec","job_status":"done","queue_duration_s":2.142122,"scheduling_latency_s":2.142144,"redis_calls":14,"redis_duration_s":0.0019069999999999998,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000233,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026813,"completed_at":"2026-09-13T10:50:06.929Z","load_balancing_strategy":"primary","db_duration_s":0.001809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.089077] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:06.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"96beff5aad2e018ed8210557","created_at":"2026-09-13T10:50:04.772Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6ef1a6114a9a233ea6b44c57dba0705","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.778Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-96beff5aad2e018ed8210557: start","job_status":"start","queue_duration_s":2.152081,"scheduling_latency_s":2.152098} gitlab # [ 284.103148] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.278Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"96beff5aad2e018ed8210557","created_at":"2026-09-13T10:50:04.772Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a6ef1a6114a9a233ea6b44c57dba0705","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.778Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-96beff5aad2e018ed8210557: done: 0.348518 sec","job_status":"done","queue_duration_s":2.152081,"scheduling_latency_s":2.152098,"redis_calls":11,"redis_duration_s":0.002144,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000267,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.062,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.062,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.27715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0004598,"exclusive_lock_hold_duration_s":0.32971652,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.348518,"completed_at":"2026-09-13T10:50:07.278Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.089754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.149129] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.280Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"da44ba673a895b2b4f014a17","created_at":"2026-09-13T10:50:04.922Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.949Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-da44ba673a895b2b4f014a17: start","job_status":"start","queue_duration_s":2.331855,"scheduling_latency_s":2.331877} gitlab # [ 284.163537] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::TimedOutBuilds::DropTimedOutWorker","jid":"da44ba673a895b2b4f014a17","created_at":"2026-09-13T10:50:04.922Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:11a5fdbfa1bf52035908acc769acf4caef1368e7f031a24ef8a306464e46866a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:04.949Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropTimedOutWorker JID-da44ba673a895b2b4f014a17: done: 0.072677 sec","job_status":"done","queue_duration_s":2.331855,"scheduling_latency_s":2.331877,"redis_calls":20,"redis_duration_s":0.007398,"redis_read_bytes":23,"redis_write_bytes":4316,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002988,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1065,"redis_queues_calls":4,"redis_queues_duration_s":0.001038,"redis_queues_read_bytes":6,"redis_queues_write_bytes":2021,"redis_queues_metadata_calls":8,"redis_queues_metadata_duration_s":0.002696,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":1004,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000676,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050688,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059,"duration_s":0.072677,"completed_at":"2026-09-13T10:50:07.353Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.207614] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"10784b625b7421b7c341c650","created_at":"2026-09-13T10:50:04.968Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1032f1bebbd474fe416ee3b1cff01c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:50:04.976Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-10784b625b7421b7c341c650: start","job_status":"start","queue_duration_s":2.379839,"scheduling_latency_s":2.37988} gitlab # [ 284.221192] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.401Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"10784b625b7421b7c341c650","created_at":"2026-09-13T10:50:04.968Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1032f1bebbd474fe416ee3b1cff01c6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:50:04.976Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-10784b625b7421b7c341c650: done: 0.044715 sec","job_status":"done","queue_duration_s":2.379839,"scheduling_latency_s":2.37988,"redis_calls":7,"redis_duration_s":0.002391,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001702,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044715,"completed_at":"2026-09-13T10:50:07.401Z","load_balancing_strategy":"primary","db_duration_s":0.002908,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.260273] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.403Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"a3c5b37968b024a4640d4501","created_at":"2026-09-13T10:50:04.986Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cf9a9ae07d940bfa70f4897eef17435","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.001Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-a3c5b37968b024a4640d4501: start","job_status":"start","queue_duration_s":2.401921,"scheduling_latency_s":2.40195} gitlab # [ 284.273444] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.434Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"a3c5b37968b024a4640d4501","created_at":"2026-09-13T10:50:04.986Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3cf9a9ae07d940bfa70f4897eef17435","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.001Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-a3c5b37968b024a4640d4501: done: 0.031548 sec","job_status":"done","queue_duration_s":2.401921,"scheduling_latency_s":2.40195,"redis_calls":12,"redis_duration_s":0.003667,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000724,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000671,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001669,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000603,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004538,"duration_s":0.031548,"completed_at":"2026-09-13T10:50:07.434Z","load_balancing_strategy":"primary","db_duration_s":0.000875,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.316094] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"e0b47b98f0a22cd6fb607187","created_at":"2026-09-13T10:50:05.006Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d90ec3fb11a8ac7465d9ae7c9a172847","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.017Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-e0b47b98f0a22cd6fb607187: start","job_status":"start","queue_duration_s":2.419002,"scheduling_latency_s":2.419027} gitlab # [ 284.331430] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.464Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"e0b47b98f0a22cd6fb607187","created_at":"2026-09-13T10:50:05.006Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d90ec3fb11a8ac7465d9ae7c9a172847","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.017Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-e0b47b98f0a22cd6fb607187: done: 0.027762 sec","job_status":"done","queue_duration_s":2.419002,"scheduling_latency_s":2.419027,"redis_calls":11,"redis_duration_s":0.0032,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000675,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001582,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000385,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000558,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038102,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.027762,"completed_at":"2026-09-13T10:50:07.464Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00078,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.375374] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b95376a5a5804eb32c9f64f0","created_at":"2026-09-13T10:50:05.024Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"048be28dbe24f8fc455f7952d154bb81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:50:05.027Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b95376a5a5804eb32c9f64f0: start","job_status":"start","queue_duration_s":2.438429,"scheduling_latency_s":2.438452} gitlab # [ 284.388962] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.476Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 284.392693] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.483Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b95376a5a5804eb32c9f64f0","created_at":"2026-09-13T10:50:05.024Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"048be28dbe24f8fc455f7952d154bb81","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:50:05.027Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b95376a5a5804eb32c9f64f0: done: 0.017084 sec","job_status":"done","queue_duration_s":2.438429,"scheduling_latency_s":2.438452,"redis_calls":7,"redis_duration_s":0.001818,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00129,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017084,"completed_at":"2026-09-13T10:50:07.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.432271] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8d6540ec8955abaa592c4625","created_at":"2026-09-13T10:50:05.031Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7bf03ac6f2d2d459edd1cccbe6a6d31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.035Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8d6540ec8955abaa592c4625: start","job_status":"start","queue_duration_s":2.453594,"scheduling_latency_s":2.453617} gitlab # [ 284.447377] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.497Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 284.451058] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.504Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8d6540ec8955abaa592c4625","created_at":"2026-09-13T10:50:05.031Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b7bf03ac6f2d2d459edd1cccbe6a6d31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.035Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8d6540ec8955abaa592c4625: done: 0.015521 sec","job_status":"done","queue_duration_s":2.453594,"scheduling_latency_s":2.453617,"redis_calls":7,"redis_duration_s":0.0018540000000000002,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001332,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009348,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015521,"completed_at":"2026-09-13T10:50:07.504Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.492583] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"39b82adc69ce1c66d810daa0","created_at":"2026-09-13T10:50:05.045Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62360434dc309a17d4464e5e4b4218d8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.049Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-39b82adc69ce1c66d810daa0: start","job_status":"start","queue_duration_s":2.456312,"scheduling_latency_s":2.456331} gitlab # [ 284.507271] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"39b82adc69ce1c66d810daa0","created_at":"2026-09-13T10:50:05.045Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62360434dc309a17d4464e5e4b4218d8","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.049Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-39b82adc69ce1c66d810daa0: done: 0.025448 sec","job_status":"done","queue_duration_s":2.456312,"scheduling_latency_s":2.456331,"redis_calls":10,"redis_duration_s":0.0028929999999999997,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001645,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001248,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015122,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00699038,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.025448,"completed_at":"2026-09-13T10:50:07.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004297,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 284.551746] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b80598cdf16b287ffe7037ac","created_at":"2026-09-13T10:50:05.066Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fb714c957abf65db29ae4454b46dc59","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.069Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b80598cdf16b287ffe7037ac: start","job_status":"start","queue_duration_s":2.463937,"scheduling_latency_s":2.463958} gitlab # [ 284.565960] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b80598cdf16b287ffe7037ac","created_at":"2026-09-13T10:50:05.066Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fb714c957abf65db29ae4454b46dc59","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.069Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b80598cdf16b287ffe7037ac: done: 0.034455 sec","job_status":"done","queue_duration_s":2.463937,"scheduling_latency_s":2.463958,"redis_calls":7,"redis_duration_s":0.002048,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.034455,"completed_at":"2026-09-13T10:50:07.567Z","load_balancing_strategy":"primary","db_duration_s":0.017839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.606847] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6b4ba3fd3b44efc2a8563072","created_at":"2026-09-13T10:50:05.080Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29299f7c0699b4847ad36b02814400f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.094Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6b4ba3fd3b44efc2a8563072: start","job_status":"start","queue_duration_s":2.475024,"scheduling_latency_s":2.475046} gitlab # [ 284.620761] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"6b4ba3fd3b44efc2a8563072","created_at":"2026-09-13T10:50:05.080Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"29299f7c0699b4847ad36b02814400f6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.094Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-6b4ba3fd3b44efc2a8563072: done: 0.022393 sec","job_status":"done","queue_duration_s":2.475024,"scheduling_latency_s":2.475046,"redis_calls":9,"redis_duration_s":0.0028780000000000003,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001528,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00135,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063924,"exclusive_lock_hold_duration_s":0.00437176,"duration_s":0.022393,"completed_at":"2026-09-13T10:50:07.592Z","load_balancing_strategy":"primary","db_duration_s":0.001801,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.660810] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.593Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerExpirationPolicyWorker","jid":"f9ec96f2803438b05264cf58","created_at":"2026-09-13T10:50:05.100Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ba5eeda814b310e5e806d97a55d657076d1f5f4bfe3538607468cb1befa65b3c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.120Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicyWorker JID-f9ec96f2803438b05264cf58: start","job_status":"start","queue_duration_s":2.473514,"scheduling_latency_s":2.473535} gitlab # [ 284.674204] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.663Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerExpirationPolicyWorker","jid":"f9ec96f2803438b05264cf58","created_at":"2026-09-13T10:50:05.100Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:ba5eeda814b310e5e806d97a55d657076d1f5f4bfe3538607468cb1befa65b3c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.120Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicyWorker JID-f9ec96f2803438b05264cf58: done: 0.069402 sec","job_status":"done","queue_duration_s":2.473514,"scheduling_latency_s":2.473535,"redis_calls":21,"redis_duration_s":0.005812,"redis_read_bytes":24,"redis_write_bytes":5387,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001222,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":724,"redis_queues_calls":2,"redis_queues_duration_s":0.000467,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3275,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.00258,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":782,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001543,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":606,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04373,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01255296,"extra.container_expiration_policy_worker.cleanup_required_count":0,"extra.container_expiration_policy_worker.cleanup_unfinished_count":0,"extra.container_expiration_policy_worker.cleanup_total_count":0,"duration_s":0.069402,"completed_at":"2026-09-13T10:50:07.663Z","load_balancing_strategy":"primary","db_duration_s":0.016122,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.719148] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.664Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"1f010290f9cbe111dae765fe","created_at":"2026-09-13T10:50:05.138Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67f79cb937b8da814f75b4095ae0224c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.152Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-1f010290f9cbe111dae765fe: start","job_status":"start","queue_duration_s":2.512311,"scheduling_latency_s":2.512333} gitlab # [ 284.734113] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"PagesDomainSslRenewalCronWorker","jid":"1f010290f9cbe111dae765fe","created_at":"2026-09-13T10:50:05.138Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67f79cb937b8da814f75b4095ae0224c","meta.root_caller_id":"Cronjob","meta.feature_category":"pages","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:cee20cd61cae5986aa71b6fa6dd981aaba3c2a4e4980220c0e15411712f7b4e3","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.152Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"PagesDomainSslRenewalCronWorker JID-1f010290f9cbe111dae765fe: done: 0.024113 sec","job_status":"done","queue_duration_s":2.512311,"scheduling_latency_s":2.512333,"redis_calls":10,"redis_duration_s":0.002652,"redis_read_bytes":9,"redis_write_bytes":1087,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000516,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001593,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":566,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000543,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":216,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036498,"duration_s":0.024113,"completed_at":"2026-09-13T10:50:07.688Z","load_balancing_strategy":"primary","db_duration_s":0.000667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.774479] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.690Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"f2aea6a62fe131d56f16b243","created_at":"2026-09-13T10:50:05.162Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f2220eea82ec69b38cff3bec9e29021","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.177Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-f2aea6a62fe131d56f16b243: start","job_status":"start","queue_duration_s":2.512774,"scheduling_latency_s":2.512792} gitlab # [ 284.788083] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.713Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"f2aea6a62fe131d56f16b243","created_at":"2026-09-13T10:50:05.162Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-4b18c95190294690","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f2220eea82ec69b38cff3bec9e29021","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:05.177Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-f2aea6a62fe131d56f16b243: done: 0.023061 sec","job_status":"done","queue_duration_s":2.512774,"scheduling_latency_s":2.512792,"redis_calls":10,"redis_duration_s":0.002956,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001503,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003902,"duration_s":0.023061,"completed_at":"2026-09-13T10:50:07.713Z","load_balancing_strategy":"primary","db_duration_s":0.002021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.831272] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.714Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"db69aa8da48620e28d200761","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-db69aa8da48620e28d200761: start","job_status":"start","queue_duration_s":0.791114,"scheduling_latency_s":0.79113} gitlab # [ 284.868486] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.727Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.873861] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.736Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"db69aa8da48620e28d200761","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-db69aa8da48620e28d200761: done: 0.021249 sec","job_status":"done","queue_duration_s":0.791114,"scheduling_latency_s":0.79113,"redis_calls":10,"redis_duration_s":0.002576,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001545,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021249,"completed_at":"2026-09-13T10:50:07.736Z","load_balancing_strategy":"primary","db_duration_s":0.003001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 284.942918] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.737Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39620f54636e0507abfa84a0","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39620f54636e0507abfa84a0: start","job_status":"start","queue_duration_s":0.81377,"scheduling_latency_s":0.813787} gitlab # [ 284.959707] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.748Z","message":"No existing merge request to be cleaned up."} gitlab # [ 284.962316] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.757Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"39620f54636e0507abfa84a0","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-39620f54636e0507abfa84a0: done: 0.019865 sec","job_status":"done","queue_duration_s":0.81377,"scheduling_latency_s":0.813787,"redis_calls":10,"redis_duration_s":0.002457,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001429,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019865,"completed_at":"2026-09-13T10:50:07.757Z","load_balancing_strategy":"primary","db_duration_s":0.002473,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.008860] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.759Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"37c9f793e646a466e1234c9d","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-37c9f793e646a466e1234c9d: start","job_status":"start","queue_duration_s":0.835298,"scheduling_latency_s":0.835319} gitlab # [ 285.021182] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.770Z","message":"No existing merge request to be cleaned up."} gitlab # [ 285.022810] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.779Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"37c9f793e646a466e1234c9d","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-37c9f793e646a466e1234c9d: done: 0.019926 sec","job_status":"done","queue_duration_s":0.835298,"scheduling_latency_s":0.835319,"redis_calls":10,"redis_duration_s":0.002497,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001047,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00145,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012647,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019926,"completed_at":"2026-09-13T10:50:07.778Z","load_balancing_strategy":"primary","db_duration_s":0.002422,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.061056] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.780Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"24bfb4d6cc93f2a14bf9f041","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-24bfb4d6cc93f2a14bf9f041: start","job_status":"start","queue_duration_s":0.856621,"scheduling_latency_s":0.856638} gitlab # [ 285.073330] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.791Z","message":"No existing merge request to be cleaned up."} gitlab # [ 285.075004] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.800Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"24bfb4d6cc93f2a14bf9f041","created_at":"2026-09-13T10:50:06.913Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-e41e518a831f49e7","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"3d9794aecd1c2c6eb73edac8015f223b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:06.923Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-24bfb4d6cc93f2a14bf9f041: done: 0.019747 sec","job_status":"done","queue_duration_s":0.856621,"scheduling_latency_s":0.856638,"redis_calls":10,"redis_duration_s":0.002497,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001431,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012638,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019747,"completed_at":"2026-09-13T10:50:07.800Z","load_balancing_strategy":"primary","db_duration_s":0.002356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.113551] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.801Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"c3c2c54bad1ea4c975b19887","created_at":"2026-09-13T10:50:07.304Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-295528db4f9540e0","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.315Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-c3c2c54bad1ea4c975b19887: start","job_status":"start","queue_duration_s":0.486563,"scheduling_latency_s":0.48658} gitlab # [ 285.128355] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.829Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropRunningWorker","jid":"c3c2c54bad1ea4c975b19887","created_at":"2026-09-13T10:50:07.304Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-295528db4f9540e0","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c1f34cecaf7af64c199dc18ea8fa612209c900930a454f797bf95dbf199c209c","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.315Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropRunningWorker JID-c3c2c54bad1ea4c975b19887: done: 0.027393 sec","job_status":"done","queue_duration_s":0.486563,"scheduling_latency_s":0.48658,"redis_calls":10,"redis_duration_s":0.003086,"redis_read_bytes":9,"redis_write_bytes":1117,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000999,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001574,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000513,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":222,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036102,"duration_s":0.027393,"completed_at":"2026-09-13T10:50:07.829Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005606,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.174745] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.830Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"22457491f777d171ae3701ec","created_at":"2026-09-13T10:50:07.316Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-295528db4f9540e0","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.329Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-22457491f777d171ae3701ec: start","job_status":"start","queue_duration_s":0.501424,"scheduling_latency_s":0.501443} gitlab # [ 285.220786] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.851Z","retry":1,"queue":"default","version":0,"store":null,"queue_namespace":"timed_out_builds","dead":false,"args":[],"class":"Ci::TimedOutBuilds::DropCancelingWorker","jid":"22457491f777d171ae3701ec","created_at":"2026-09-13T10:50:07.316Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-295528db4f9540e0","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"Ci::TimedOutBuilds::DropTimedOutWorker","correlation_id":"c4ff6f381314ca8a968bb8b288208ad5","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:23125b5131e5285ca63ba8858fc6fb370306fe43f055d056b2a837c90773932d","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.329Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TimedOutBuilds::DropCancelingWorker JID-22457491f777d171ae3701ec: done: 0.021035 sec","job_status":"done","queue_duration_s":0.501424,"scheduling_latency_s":0.501443,"redis_calls":10,"redis_duration_s":0.002539,"redis_read_bytes":9,"redis_write_bytes":1133,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000529,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001531,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":226,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012564,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035346,"duration_s":0.021035,"completed_at":"2026-09-13T10:50:07.851Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001374,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.304316] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.853Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"c9cba548b2d4349b43fea1f6","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-c9cba548b2d4349b43fea1f6: start","job_status":"start","queue_duration_s":0.206946,"scheduling_latency_s":0.206963} gitlab # [ 285.324637] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"c9cba548b2d4349b43fea1f6","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-c9cba548b2d4349b43fea1f6: done: 0.037564 sec","job_status":"done","queue_duration_s":0.206946,"scheduling_latency_s":0.206963,"redis_calls":13,"redis_duration_s":0.003339,"redis_read_bytes":12,"redis_write_bytes":1785,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000491,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":365,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":559,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001579,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":861,"db_count":5,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024882,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033194,"duration_s":0.037564,"completed_at":"2026-09-13T10:50:07.890Z","load_balancing_strategy":"primary","db_duration_s":0.005927,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.372478] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"69ff9c41b7d2dd725eb59618","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-69ff9c41b7d2dd725eb59618: start","job_status":"start","queue_duration_s":0.246154,"scheduling_latency_s":0.246173} gitlab # [ 285.385991] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"69ff9c41b7d2dd725eb59618","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-69ff9c41b7d2dd725eb59618: done: 0.030955 sec","job_status":"done","queue_duration_s":0.246154,"scheduling_latency_s":0.246173,"redis_calls":10,"redis_duration_s":0.0026360000000000003,"redis_read_bytes":8,"redis_write_bytes":1310,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":449,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001575,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":861,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030955,"completed_at":"2026-09-13T10:50:07.923Z","load_balancing_strategy":"primary","db_duration_s":0.005261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.425772] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"f7f217d85b8f10399eac1db1","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-f7f217d85b8f10399eac1db1: start","job_status":"start","queue_duration_s":0.278576,"scheduling_latency_s":0.278594} gitlab # [ 285.439258] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"f7f217d85b8f10399eac1db1","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-f7f217d85b8f10399eac1db1: done: 0.031095 sec","job_status":"done","queue_duration_s":0.278576,"scheduling_latency_s":0.278594,"redis_calls":10,"redis_duration_s":0.002729,"redis_read_bytes":8,"redis_write_bytes":1310,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":449,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001614,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":861,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031095,"completed_at":"2026-09-13T10:50:07.955Z","load_balancing_strategy":"primary","db_duration_s":0.005192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 285.479061] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"cee53faa71d3ab80fb680d6f","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-cee53faa71d3ab80fb680d6f: start","job_status":"start","queue_duration_s":0.31109,"scheduling_latency_s":0.311108} gitlab # [ 285.492570] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:07.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"queue_namespace":"container_repository","class":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker","args":[],"jid":"cee53faa71d3ab80fb680d6f","created_at":"2026-09-13T10:50:07.635Z","trace_propagation_headers":{"sentry-trace":"81907fd592914734bb6145c29352ffcb-597f997b971848f4","baggage":"sentry-trace_id=81907fd592914734bb6145c29352ffcb,sentry-release=Unknown"},"meta.caller_id":"ContainerExpirationPolicyWorker","correlation_id":"3b7e4acc93c358db1b3c86ae27a7c480","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:50:07.646Z","job_size_bytes":2,"pid":2030,"sidekiq_tid":"maoi","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerExpirationPolicies::CleanupContainerRepositoryWorker JID-cee53faa71d3ab80fb680d6f: done: 0.035023 sec","job_status":"done","queue_duration_s":0.31109,"scheduling_latency_s":0.311108,"redis_calls":10,"redis_duration_s":0.0020109999999999998,"redis_read_bytes":8,"redis_write_bytes":1310,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000779,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":449,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":861,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021897,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035023,"completed_at":"2026-09-13T10:50:07.992Z","load_balancing_strategy":"primary","db_duration_s":0.009073,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.213646] redis[783]: 10 changes in 300 seconds. Saving... gitlab # [ 310.220080] redis[783]: Background saving started by pid 3305 gitlab # [ 310.224819] redis[3305]: BGSAVE done, 510 keys saved, 0 keys skipped, 128092 bytes written. gitlab # [ 310.229484] redis[3305]: DB saved on disk gitlab # [ 310.231721] redis[3305]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 310.314850] redis[783]: Background saving terminated with success gitlab # [ 311.911248] postgres[856]: [856] LOG: checkpoint starting: time gitlab # [ 313.332731] gitaly[891]: time="2026-09-13T10:50:38.206Z" level=info msg="burden monitor stats" pid=891 gitlab # [ 318.998118] postgres[3323]: [3323] ERROR: relation "table_without_models" does not exist gitlab # [ 319.000490] postgres[3323]: [3323] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 325.255922] runuser[2979]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 325.257775] setup-gitlab-start[2952]: + systemctl restart gitlab-sidekiq.service gitlab # [ 325.287925] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 325.291650] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:50.166Z","message":"Shutting down"} gitlab # [ 325.293445] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:50.167Z","message":"Scheduler exiting..."} gitlab # [ 325.297702] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:50.172Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 325.302873] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:50.173Z","message":"Scheduler exiting..."} gitlab # [ 325.803753] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:50.678Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 328.808296] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:53.682Z","message":"Bye!"} gitlab # [ 328.849725] sidekiq-cluster[2030]: {"severity":"INFO","time":"2026-09-13T10:50:53.724Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2030,"worker_id":"sidekiq_0","memwd_rss_bytes":957259776,"retry":0} gitlab # [ 329.415908] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 329.419238] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 329.421691] systemd[1]: gitlab-sidekiq.service: Consumed 1min 52.799s CPU time over 3min 23.572s wall clock time, 949.1M memory peak, 4.5M read from disk. gitlab # [ 329.427836] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 329.431299] setup-gitlab-start[2952]: + mkdir -p /var/lib/secrets gitlab # [ 329.434990] setup-gitlab-start[2952]: + echo nixbot-test-token-of-20-chars gitlab # [ 329.437931] setup-gitlab-start[2952]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 329.441016] setup-gitlab-start[2952]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 330.291468] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:50:55 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 804 gitlab # [ 331.802050] gitaly[891]: time="2026-09-13T10:50:56.676Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:06.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:56.675" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 331.820275] gitaly[891]: time="2026-09-13T10:50:56.676Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:06.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:56.675" grpc.time_ms=0.588 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.255206] gitaly[891]: time="2026-09-13T10:50:57.130Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.130" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.278021] gitaly[891]: time="2026-09-13T10:50:57.130Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.130" grpc.time_ms=0.574 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.332411] gitaly[891]: time="2026-09-13T10:50:57.207Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP diskcache=a7b288a1-f460-4523-9ddd-99d5a3f99a11 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:51:07.031" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.152" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.344655] gitaly[891]: time="2026-09-13T10:50:57.208Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=32 command.spawn_token_fork_ms=5 command.system_time_ms=7 command.user_time_ms=18 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:51:07.031" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.152" grpc.time_ms=55.158 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.368932] gitaly[891]: time="2026-09-13T10:50:57.244Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.122" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.242" pid=891 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 332.380888] gitaly[891]: time="2026-09-13T10:50:57.244Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.122" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.242" grpc.time_ms=1.538 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.405189] gitaly[891]: time="2026-09-13T10:50:57.280Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:07.141" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.262" grpc.time_ms=17.701 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.626637] gitaly[891]: time="2026-09-13T10:50:57.501Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.500" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.658035] gitaly[891]: time="2026-09-13T10:50:57.502Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.500" grpc.time_ms=1.031 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.693161] gitaly[891]: time="2026-09-13T10:50:57.568Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP diskcache=d109b704-17e0-48b6-b4da-4307456b65c8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:51:07.017" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.508" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.721429] gitaly[891]: time="2026-09-13T10:50:57.568Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=37 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=41 command.spawn_token_fork_ms=3 command.system_time_ms=31 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-13T10:51:07.017" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.508" grpc.time_ms=60.38 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.751791] gitaly[891]: time="2026-09-13T10:50:57.584Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.583" pid=891 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 332.769135] gitaly[891]: time="2026-09-13T10:50:57.584Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.093" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.583" grpc.time_ms=0.746 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.784160] gitaly[891]: time="2026-09-13T10:50:57.607Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=7 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:07.104" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.595" grpc.time_ms=12.459 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.845203] gitaly[891]: time="2026-09-13T10:50:57.718Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:51:07.046" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.717" grpc.time_ms=0.381 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.877551] gitaly[891]: time="2026-09-13T10:50:57.751Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:51:07.080" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:57.750" grpc.time_ms=0.522 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.557288] gitaly[891]: time="2026-09-13T10:50:59.430Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=6 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ETWV58EP2D9X4XRQMNP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:09.290" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:50:59.412" grpc.time_ms=17.67 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.782794] sidekiq-cluster[3389]: {"severity":"INFO","time":"2026-09-13T10:50:59.614Z","message":"Starting cluster with 1 processes"} gitlab # [ 335.199104] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:00 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 4876 gitlab # [ 335.205127] setup-gitlab-start[3401]: {"id":1,"message":null,"error":null} gitlab # [ 340.965445] gitaly[891]: time="2026-09-13T10:51:05.840Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68RGR710BSZ2QMHHW0EZ7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:15.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:05.823" grpc.time_ms=16.508 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.884318] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:06 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 1644 gitlab # [ 341.887900] setup-gitlab-start[2952]: + mkdir -p /tmp/test-flake gitlab # [ 341.892355] setup-gitlab-start[2952]: + cd /tmp/test-flake gitlab # [ 341.892567] setup-gitlab-start[2952]: + git init -b master gitlab # [ 341.922184] setup-gitlab-start[3503]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 341.929947] setup-gitlab-start[2952]: + git config user.name test gitlab # [ 341.936256] setup-gitlab-start[2952]: + git config user.email test@example.com gitlab # [ 341.942766] setup-gitlab-start[2952]: + cat gitlab # [ 341.948261] setup-gitlab-start[2952]: + git add flake.nix gitlab # [ 341.957051] setup-gitlab-start[2952]: + git commit -m 'initial commit' gitlab # [ 341.987641] setup-gitlab-start[3508]: [master (root-commit) 541e2c7] initial commit gitlab # [ 341.992635] setup-gitlab-start[3508]: 1 file changed, 10 insertions(+) gitlab # [ 341.995365] setup-gitlab-start[3508]: create mode 100644 flake.nix gitlab # [ 341.998170] setup-gitlab-start[2952]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 342.003846] setup-gitlab-start[2952]: + git push -u origin master gitlab # [ 342.119828] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:06 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 70 gitlab # [ 342.295788] gitaly[891]: time="2026-09-13T10:51:07.169Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68TBMY6PWF0ZKVX7DWASY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:07.152" grpc.time_ms=17.444 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 342.338115] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:07 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 174 gitlab # [ 342.643523] gitaly[891]: time="2026-09-13T10:51:07.518Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:17.010" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:07.501" grpc.time_ms=17.19 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.706416] gitaly[891]: time="2026-09-13T10:51:07.581Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:51:37.063" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:51:07.563" grpc.time_ms=17.561 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.749388] gitaly[891]: time="2026-09-13T10:51:07.621Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG duration_ms=230ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 342.790617] gitaly[891]: time="2026-09-13T10:51:07.665Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG duration_ms=42ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 342.800369] gitaly[891]: time="2026-09-13T10:51:07.666Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:07.387" grpc.time_ms=278.245 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 342.858482] gitaly[891]: time="2026-09-13T10:51:07.733Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:07.732" grpc.time_ms=1.471 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 342.921717] gitaly[891]: time="2026-09-13T10:51:07.796Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:07.795" grpc.time_ms=1.054 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 342.975384] gitaly[891]: time="2026-09-13T10:51:07.850Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:07.849" grpc.time_ms=0.654 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 343.166325] gitaly[891]: time="2026-09-13T10:51:08.041Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:51:18.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:08.024" grpc.time_ms=16.995 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.220411] gitaly[891]: time="2026-09-13T10:51:08.095Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=25 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=5 command.system_time_ms=15 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:51:18.039" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:08.060" grpc.time_ms=34.573 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.256839] gitaly[891]: time="2026-09-13T10:51:08.132Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG duration_ms=220ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 343.265851] gitaly[891]: time="2026-09-13T10:51:08.132Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:07.910" grpc.time_ms=222.426 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 343.302339] gitaly[891]: time="2026-09-13T10:51:08.176Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG diskcache=0b20e571-7c4b-4278-b8fc-eb0fb0fca9d5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:07.275" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 343.327757] gitaly[891]: time="2026-09-13T10:51:08.178Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=342 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13639 command.oublock=32 command.real_time_ms=863 command.spawn_token_fork_ms=2 command.system_time_ms=201 command.user_time_ms=141 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:07.275" grpc.time_ms=901.652 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 343.354574] systemd[1]: Finished setup-gitlab.service. gitlab # [ 343.356499] setup-gitlab-start[3512]: To http://gitlab/root/test-flake.git gitlab # [ 343.357894] setup-gitlab-start[3512]: * [new branch] master -> master gitlab # [ 343.359306] setup-gitlab-start[3512]: branch 'master' set up to track 'origin/master'. gitlab # [ 343.361503] gitlab-workhorse[815]: time="2026-09-13T10:51:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 343.364400] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:08 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 990 gitlab # [ 343.367332] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 344.70 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 344.684151] nixbot[3588]: {"ts": "2026-09-13T10:51:09.558Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 344.711915] nixbot[3588]: {"ts": "2026-09-13T10:51:09.586Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 344.755452] nixbot[3588]: {"ts": "2026-09-13T10:51:09.630Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 344.763743] nixbot[3588]: {"ts": "2026-09-13T10:51:09.638Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 344.771311] nixbot[3588]: {"ts": "2026-09-13T10:51:09.646Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 344.783615] nixbot[3588]: {"ts": "2026-09-13T10:51:09.658Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 344.790850] nixbot[3588]: {"ts": "2026-09-13T10:51:09.665Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 344.794829] nixbot[3588]: {"ts": "2026-09-13T10:51:09.669Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 344.805880] nixbot[3588]: {"ts": "2026-09-13T10:51:09.680Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 344.812727] nixbot[3588]: {"ts": "2026-09-13T10:51:09.684Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 344.819509] nixbot[3588]: {"ts": "2026-09-13T10:51:09.694Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 344.831229] nixbot[3588]: {"ts": "2026-09-13T10:51:09.706Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 344.837993] nixbot[3588]: {"ts": "2026-09-13T10:51:09.711Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 344.844824] nixbot[3588]: {"ts": "2026-09-13T10:51:09.714Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 344.851726] nixbot[3588]: {"ts": "2026-09-13T10:51:09.721Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 344.858574] nixbot[3588]: {"ts": "2026-09-13T10:51:09.724Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 344.865242] nixbot[3588]: {"ts": "2026-09-13T10:51:09.726Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 344.871936] nixbot[3588]: {"ts": "2026-09-13T10:51:09.729Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 344.878048] nixbot[3588]: {"ts": "2026-09-13T10:51:09.736Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 344.884143] nixbot[3588]: {"ts": "2026-09-13T10:51:09.740Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 344.890220] nixbot[3588]: {"ts": "2026-09-13T10:51:09.742Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 344.896558] nixbot[3588]: {"ts": "2026-09-13T10:51:09.750Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 344.902233] nixbot[3588]: {"ts": "2026-09-13T10:51:09.753Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 344.907439] nixbot[3588]: {"ts": "2026-09-13T10:51:09.756Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 344.912512] nixbot[3588]: {"ts": "2026-09-13T10:51:09.759Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 344.917583] nixbot[3588]: {"ts": "2026-09-13T10:51:09.762Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 344.922918] nixbot[3588]: {"ts": "2026-09-13T10:51:09.765Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 344.927960] nixbot[3588]: {"ts": "2026-09-13T10:51:09.767Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 344.932545] nixbot[3588]: {"ts": "2026-09-13T10:51:09.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 344.936913] nixbot[3588]: {"ts": "2026-09-13T10:51:09.786Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 344.941158] nixbot[3588]: {"ts": "2026-09-13T10:51:09.792Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 344.945271] nixbot[3588]: {"ts": "2026-09-13T10:51:09.795Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 344.949406] nixbot[3588]: {"ts": "2026-09-13T10:51:09.797Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 344.953314] nixbot[3588]: {"ts": "2026-09-13T10:51:09.803Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 344.957033] nixbot[3588]: {"ts": "2026-09-13T10:51:09.805Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 344.960486] nixbot[3588]: {"ts": "2026-09-13T10:51:09.812Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 345.400269] nixbot[3588]: INFO: Started server process [3588] gitlab # [ 345.402961] nixbot[3588]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 345.429058] nixbot[3588]: INFO: ::ffff:127.0.0.1:38718 - "GET /health HTTP/1.1" 200 OK gitlab # [ 345.437321] systemd[1]: Started nixbot CI service. gitlab # [ 345.442163] systemd[1]: Reached target Multi-User System. gitlab # [ 345.445508] systemd[1]: Startup finished in 789ms (kernel) + 5.294s (initrd) + 5min 39.362s (userspace) = 5min 45.445s. gitlab # [ 345.811056] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:10 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 382 gitlab # [ 345.819622] nixbot[3588]: {"ts": "2026-09-13T10:51:10.687Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 345.850689] nixbot[3588]: {"ts": "2026-09-13T10:51:10.725Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 345.948264] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 85 gitlab # [ 345.954756] nixbot[3588]: {"ts": "2026-09-13T10:51:10.824Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 345.961799] nixbot[3588]: {"ts": "2026-09-13T10:51:10.826Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 346.088911] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:10 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 134 gitlab # [ 346.095392] nixbot[3588]: {"ts": "2026-09-13T10:51:10.964Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab: (finished: waiting for unit nixbot.service, in 2.13 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 346.418646] nixbot[3588]: INFO: ::ffff:127.0.0.1:38728 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 346.544195] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:11 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 443 gitlab # [ 346.552194] nixbot[3588]: {"ts": "2026-09-13T10:51:11.416Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 346.607098] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 131 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.19 seconds) (finished: subtest: project discovered and webhook registered, in 0.19 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 346.734168] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 169 gitlab # [ 346.736638] nixbot[3588]: {"ts": "2026-09-13T10:51:11.611Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 346.832344] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:11 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 86 gitlab # [ 346.839937] nixbot[3588]: {"ts": "2026-09-13T10:51:11.707Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 346.926600] gitaly[891]: time="2026-09-13T10:51:11.800Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68YZ6FWDA1NVR4WCTWH8H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-13T10:51:41.273" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:11.774" grpc.time_ms=26.1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 346.971570] gitaly[891]: time="2026-09-13T10:51:11.820Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68YZ6FWDA1NVR4WCTWH8H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:41.318" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:11.818" grpc.time_ms=1.43 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 346.996273] gitaly[891]: time="2026-09-13T10:51:11.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68YZ6FWDA1NVR4WCTWH8H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-13T10:51:21.005" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:11.825" grpc.time_ms=9.887 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.016151] nixbot[3588]: {"ts": "2026-09-13T10:51:11.873Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 347.019302] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 152 gitlab # [ 347.254881] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:12 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 246 gitlab # [ 347.257660] nixbot[3588]: {"ts": "2026-09-13T10:51:12.132Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 347.264584] nixbot[3588]: {"ts": "2026-09-13T10:51:12.139Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 347.422696] gitaly[891]: time="2026-09-13T10:51:12.297Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D68ZEFE7P2DCCZYZQ4KFKG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.280" grpc.time_ms=17.115 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 347.441682] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 91 gitlab # [ 347.690489] gitaly[891]: time="2026-09-13T10:51:12.565Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZHDY3CYAYGB8CJZQQF6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.545" pid=891 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 347.723245] gitaly[891]: time="2026-09-13T10:51:12.565Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZHDY3CYAYGB8CJZQQF6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.545" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 347.756117] gitlab-workhorse[815]: time="2026-09-13T10:51:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 347.759644] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 265 gitlab # [ 347.764059] gitaly[891]: time="2026-09-13T10:51:12.566Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=4 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZHDY3CYAYGB8CJZQQF6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.545" grpc.time_ms=20.88 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 347.884255] gitaly[891]: time="2026-09-13T10:51:12.759Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZTRJT30MTETRWZCAQPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.724" pid=891 remote_ip="2001:db8:1::1" request_sha=492e789ca99033e9b047ee5d6f63ba60d0b5ffda response_bytes=222 span.kind=server system=grpc gitlab # [ 347.915919] gitaly[891]: time="2026-09-13T10:51:12.759Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=26 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZTRJT30MTETRWZCAQPZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.724" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 347.949713] gitlab-workhorse[815]: time="2026-09-13T10:51:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=222 gitlab # [ 347.953224] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 222 "" "git/2.55.0" 159 gitlab # [ 347.960845] gitaly[891]: time="2026-09-13T10:51:12.760Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=26 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68ZTRJT30MTETRWZCAQPZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.724" grpc.time_ms=35.006 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.069286] gitaly[891]: time="2026-09-13T10:51:12.944Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D6901YM4REM9FEC3165HYZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:12.926" grpc.time_ms=17.498 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.106169] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 114 gitlab # [ 348.227982] gitaly[891]: time="2026-09-13T10:51:13.103Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6905K9J8N1P3H31MWG8KB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.084" pid=891 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 348.246487] gitaly[891]: time="2026-09-13T10:51:13.103Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6905K9J8N1P3H31MWG8KB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.084" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.270329] gitlab-workhorse[815]: time="2026-09-13T10:51:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 348.273009] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 157 gitlab # [ 348.275580] gitaly[891]: time="2026-09-13T10:51:13.104Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6905K9J8N1P3H31MWG8KB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.084" grpc.time_ms=19.47 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.440591] gitaly[891]: time="2026-09-13T10:51:13.315Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M2D690DP04D5GB0SASKZSR2C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.299" grpc.time_ms=16.109 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.480742] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:13 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 109 gitlab # [ 348.564368] gitaly[891]: time="2026-09-13T10:51:13.439Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D690H65WWFGCVQMK6H5YSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.404" pid=891 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 348.596759] gitaly[891]: time="2026-09-13T10:51:13.439Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=25 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690H65WWFGCVQMK6H5YSN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.404" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.626993] gitlab-workhorse[815]: time="2026-09-13T10:51:13Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 348.630576] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:13 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 122 gitlab # [ 348.634226] gitaly[891]: time="2026-09-13T10:51:13.440Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=29 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=30 command.spawn_token_fork_ms=2 command.system_time_ms=25 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690H65WWFGCVQMK6H5YSN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:13.404" grpc.time_ms=35.652 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.780697] gitaly[891]: time="2026-09-13T10:51:13.655Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:43.053" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:13.653" grpc.time_ms=1.469 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 348.855477] gitaly[891]: time="2026-09-13T10:51:13.730Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:51:43.110" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:13.710" grpc.time_ms=19.553 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 348.909432] gitaly[891]: time="2026-09-13T10:51:13.784Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:51:23.092" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:13.762" grpc.time_ms=21.098 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 348.955545] gitaly[891]: time="2026-09-13T10:51:13.830Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=8 component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:51:23.137" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:13.808" grpc.time_ms=22.305 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.104217] gitaly[891]: time="2026-09-13T10:51:13.977Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:51:23.006" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:13.977" grpc.time_ms=0.667 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.139609] gitaly[891]: time="2026-09-13T10:51:14.014Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:51:23.024" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:13.996" grpc.time_ms=18.422 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.226375] gitaly[891]: time="2026-09-13T10:51:14.101Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:14.097" grpc.time_ms=4.311 pid=891 span.kind=server system=grpc gitlab # [ 349.289038] gitaly[891]: time="2026-09-13T10:51:14.164Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:51:14.163" grpc.time_ms=0.971 pid=891 span.kind=server system=grpc gitlab # [ 349.328129] gitaly[891]: time="2026-09-13T10:51:14.203Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS diskcache=399b04d0-0d35-415c-bb07-701786acedac grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:51:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:14.020" pid=891 ref=refs/keep-around/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c remote_ip="2001:db8:1::1" revision=541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c span.kind=server system=grpc username=root gitlab # [ 349.362113] gitaly[891]: time="2026-09-13T10:51:14.203Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=150 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5421 command.oublock=8 command.real_time_ms=173 command.spawn_token_fork_ms=2 command.system_time_ms=82 command.user_time_ms=68 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:51:24.009" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:51:14.020" grpc.time_ms=182.93 pid=891 ref=refs/keep-around/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c remote_ip="2001:db8:1::1" revision=541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c span.kind=server system=grpc username=root gitlab # [ 349.392236] gitaly[891]: time="2026-09-13T10:51:14.212Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:44.010" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:14.211" grpc.time_ms=0.628 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 349.598901] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:14 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 967 gitlab # [ 349.608069] nixbot[3588]: {"ts": "2026-09-13T10:51:14.474Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 349.617236] nixbot[3588]: {"ts": "2026-09-13T10:51:14.488Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-541e2c75d6ab-0"} gitlab # [ 349.899368] nixbot[3588]: {"ts": "2026-09-13T10:51:14.774Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-541e2c75d6ab-0", "argv0": "bwrap"} gitlab # [ 350.087749] systemd[1]: Started Nix Daemon. gitlab # [ 350.198809] nix-daemon[3760]: accepted connection from pid 3751, user nixbot gitlab # [ 354.476776] nix-daemon[3760]: accepted connection from pid 3800, user nixbot gitlab # [ 354.505017] nix-daemon[3760]: accepted connection from pid 3751, user nixbot gitlab # [ 359.116587] nix-daemon[3760]: accepted connection from pid 3751, user nixbot gitlab # [ 359.328957] gitaly[891]: time="2026-09-13T10:51:24.203Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69B1FAAMFWGX247KMQECH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:54.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:24.201" grpc.time_ms=1.122 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 359.410226] gitaly[891]: time="2026-09-13T10:51:24.285Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D69B1FAAMFWGX247KMQECH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:51:34.259" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:24.270" grpc.time_ms=14.719 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 359.461221] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 254 gitlab # [ 359.470075] nixbot[3588]: {"ts": "2026-09-13T10:51:24.338Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 359.687199] gitaly[891]: time="2026-09-13T10:51:24.562Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69B9S6ERKH8CTK4P8RG1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:54.360" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:24.560" grpc.time_ms=1.15 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 359.728386] gitaly[891]: time="2026-09-13T10:51:24.597Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D69B9S6ERKH8CTK4P8RG1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:51:34.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:24.588" grpc.time_ms=9.235 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 359.774166] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 303 gitlab # [ 359.782283] nixbot[3588]: {"ts": "2026-09-13T10:51:24.650Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 359.847673] gitaly[891]: time="2026-09-13T10:51:24.722Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69BKP2DHDW8122Z83DCTM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:51:54.520" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:51:24.721" grpc.time_ms=1.027 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 359.909820] gitaly[891]: time="2026-09-13T10:51:24.785Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D69BKP2DHDW8122Z83DCTM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:51:34.193" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:51:24.773" grpc.time_ms=11.316 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 360.005538] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 217 gitlab # [ 360.010702] nixbot[3588]: {"ts": "2026-09-13T10:51:24.882Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 360.638445] gitaly[891]: time="2026-09-13T10:51:25.512Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D69CAT7AKTRBXV1Q0TNB4H grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.494" grpc.time_ms=18.119 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 360.675133] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 110 gitlab # [ 360.747035] gitaly[891]: time="2026-09-13T10:51:25.622Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CEBQK3KBA2VF1SE7W7N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.604" pid=891 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 360.780631] gitaly[891]: time="2026-09-13T10:51:25.622Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CEBQK3KBA2VF1SE7W7N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.604" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 360.811851] gitlab-workhorse[815]: time="2026-09-13T10:51:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 360.815433] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 107 gitlab # [ 360.818933] gitaly[891]: time="2026-09-13T10:51:25.626Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CEBQK3KBA2VF1SE7W7N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.604" grpc.time_ms=18.315 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 360.886166] gitaly[891]: time="2026-09-13T10:51:25.761Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D69CKRGZJA0MQMH4PQYT89 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.751" grpc.time_ms=10.183 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 360.924587] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 72 gitlab # [ 360.991451] gitaly[891]: time="2026-09-13T10:51:25.866Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CP3Y48NP7QCT98TFCZR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.847" pid=891 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 361.019472] gitaly[891]: time="2026-09-13T10:51:25.866Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CP3Y48NP7QCT98TFCZR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.847" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 361.043915] gitlab-workhorse[815]: time="2026-09-13T10:51:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 361.047430] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:51:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 104 gitlab # [ 361.050397] gitaly[891]: time="2026-09-13T10:51:25.867Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D69CP3Y48NP7QCT98TFCZR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:51:25.847" grpc.time_ms=19.982 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 376.148154] sidekiq-cluster[3454]: 2026-09-13T10:51:41.021Z pid=3454 tid=yqe INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3454, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>430469120} gitlab # [ 376.454241] sidekiq-cluster[3454]: W, [2026-09-13T10:51:41.329270 #3454] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 376.518771] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:51:41.393Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 379.923351] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:51:44.797Z","message":"GitLab reliable fetch activated!"} gitlab # [ 379.923727] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:51:44.797Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 418.015797] postgres[4312]: [4312] ERROR: relation "table_without_models" does not exist gitlab # [ 418.016459] postgres[4312]: [4312] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 422.806939] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:27.681Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 422.810128] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:27.684Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 422.812148] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:27.686Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 422.813863] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:27.686Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 422.815834] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:27.689Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 76.48 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 423.377859] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:28 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 192 gitlab # [ 423.563121] gitaly[891]: time="2026-09-13T10:52:28.437Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9PXQ1Q3E38G0CEHVXZT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:28.416" grpc.time_ms=20.256 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 423.599898] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:28 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 203 "" "git/2.55.0" 184 gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 137.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 424.116165] gitaly[891]: time="2026-09-13T10:52:28.991Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-13T10:52:58.475" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-13T10:52:28.976" grpc.time_ms=14.266 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.158530] gitaly[891]: time="2026-09-13T10:52:29.014Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M duration_ms=159ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 424.216461] gitaly[891]: time="2026-09-13T10:52:29.091Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M duration_ms=75ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 424.225855] gitaly[891]: time="2026-09-13T10:52:29.092Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:28.848" grpc.time_ms=244.053 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 424.281884] gitaly[891]: time="2026-09-13T10:52:29.157Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:29.155" grpc.time_ms=1.4 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 424.325189] gitaly[891]: time="2026-09-13T10:52:29.199Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:29.195" grpc.time_ms=2.765 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 424.403844] gitaly[891]: time="2026-09-13T10:52:29.278Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:29.276" grpc.time_ms=1.191 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 424.598064] gitaly[891]: time="2026-09-13T10:52:29.472Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M duration_ms=136ns method=POST pid=891 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 424.607635] gitaly[891]: time="2026-09-13T10:52:29.472Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:29.333" grpc.time_ms=139.178 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 424.642156] gitaly[891]: time="2026-09-13T10:52:29.517Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M diskcache=f622578f-b5d6-40ed-ab40-5283fab26165 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:28.724" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 424.667434] gitaly[891]: time="2026-09-13T10:52:29.517Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=384 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13598 command.oublock=32 command.real_time_ms=785 command.spawn_token_fork_ms=2 command.system_time_ms=231 command.user_time_ms=153 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:28.724" grpc.time_ms=793.191 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # 541e2c7..3c4e47e master -> master gitlab # [ 424.694108] gitlab-workhorse[815]: time="2026-09-13T10:52:29Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 424.697210] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:29 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1024 gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.62 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.14 seconds) gitlab # [ 424.857800] nixbot[3588]: INFO: ::ffff:127.0.0.1:42704 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 425.649898] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:30.523Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"35693490cb64dd7078b5c323","created_at":"2026-09-13T10:50:57.307Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:57.392Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-35693490cb64dd7078b5c323: start","job_status":"start","queue_duration_s":93.130868,"scheduling_latency_s":93.130924} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 425.912004] nixbot[3588]: INFO: ::ffff:127.0.0.1:42708 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.05 seconds) gitlab # [ 425.928767] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:30.802Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"35693490cb64dd7078b5c323","created_at":"2026-09-13T10:50:57.307Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:57.392Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-35693490cb64dd7078b5c323: done: 0.279267 sec","job_status":"done","queue_duration_s":93.130868,"scheduling_latency_s":93.130924,"redis_calls":23,"redis_duration_s":0.008971,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.00416,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002975,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.128177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14927822,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.279267,"completed_at":"2026-09-13T10:52:30.802Z","load_balancing_strategy":"primary","db_duration_s":0.096602,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.006433] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:30.806Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"35c1c21292fdcbf89b166ae0","created_at":"2026-09-13T10:50:57.469Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:57.486Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-35c1c21292fdcbf89b166ae0: start","job_status":"start","queue_duration_s":93.319475,"scheduling_latency_s":93.319517} gitlab # [ 426.123781] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:30.998Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"35c1c21292fdcbf89b166ae0","created_at":"2026-09-13T10:50:57.469Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:57.486Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-35c1c21292fdcbf89b166ae0: done: 0.192093 sec","job_status":"done","queue_duration_s":93.319475,"scheduling_latency_s":93.319517,"redis_calls":14,"redis_duration_s":0.005449,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00107,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002638,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001741,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.141405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.14566346,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.192093,"completed_at":"2026-09-13T10:52:30.998Z","load_balancing_strategy":"primary","db_duration_s":0.037575,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.218667] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.001Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"905eba92870686eb658c765f","created_at":"2026-09-13T10:50:59.283Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.291Z","job_size_bytes":5,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-905eba92870686eb658c765f: start","job_status":"start","queue_duration_s":91.70936,"scheduling_latency_s":91.709391} gitlab # [ 426.375187] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.249Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"905eba92870686eb658c765f","created_at":"2026-09-13T10:50:59.283Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.291Z","job_size_bytes":5,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-905eba92870686eb658c765f: done: 0.248905 sec","job_status":"done","queue_duration_s":91.70936,"scheduling_latency_s":91.709391,"redis_calls":12,"redis_duration_s":0.003668,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001128,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002021,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000519,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.187066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000359,"duration_s":0.248905,"completed_at":"2026-09-13T10:52:31.249Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047017,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.482651] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.252Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"2872aed6fca10ebf0dbed70e","created_at":"2026-09-13T10:50:59.332Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.338Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-2872aed6fca10ebf0dbed70e: start","job_status":"start","queue_duration_s":91.913306,"scheduling_latency_s":91.913327} gitlab # [ 426.778914] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.652Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"2872aed6fca10ebf0dbed70e","created_at":"2026-09-13T10:50:59.332Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.338Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-2872aed6fca10ebf0dbed70e: done: 0.40016 sec","job_status":"done","queue_duration_s":91.913306,"scheduling_latency_s":91.913327,"redis_calls":11,"redis_duration_s":0.003541,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00075,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002126,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.351,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.351,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.043,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.292643,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079116,"duration_s":0.40016,"completed_at":"2026-09-13T10:52:31.652Z","load_balancing_strategy":"primary","db_duration_s":0.117076,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.886382] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.655Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"cd4045cba0b0d3c6413710b8","created_at":"2026-09-13T10:50:59.343Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.348Z","job_size_bytes":89,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-cd4045cba0b0d3c6413710b8: start","job_status":"start","queue_duration_s":92.306595,"scheduling_latency_s":92.306629} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.05 seconds) gitlab # [ 426.911717] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.717Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"cd4045cba0b0d3c6413710b8","created_at":"2026-09-13T10:50:59.343Z","trace_propagation_headers":{"sentry-trace":"2659b42656204b3098a0466953bdaa00-4cac49c43abf4ea9","baggage":"sentry-trace_id=2659b42656204b3098a0466953bdaa00,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68ETWV58EP2D9X4XRQMNP","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-13T10:50:59.348Z","job_size_bytes":89,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-cd4045cba0b0d3c6413710b8: done: 0.062435 sec","job_status":"done","queue_duration_s":92.306595,"scheduling_latency_s":92.306629,"redis_calls":11,"redis_duration_s":0.003653,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000951,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002029,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000673,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04208,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066388,"duration_s":0.062435,"completed_at":"2026-09-13T10:52:31.717Z","load_balancing_strategy":"primary","db_duration_s":0.008782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.963734] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.720Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"73d47bb01e85f897a8dd1cf0","created_at":"2026-09-13T10:51:07.988Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-bca2a959a34c4e41","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:51:07.997Z","job_size_bytes":185,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-73d47bb01e85f897a8dd1cf0: start","job_status":"start","queue_duration_s":83.72227,"scheduling_latency_s":83.722298} gitlab # [ 426.983719] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:31.743Z","message":"changes: \"0000000000000000000000000000000000000000 541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c refs/heads/master\\n\""} gitlab # [ 426.986983] nixbot[3588]: INFO: ::ffff:127.0.0.1:42724 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 426.989497] gitaly[891]: time="2026-09-13T10:52:31.793Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:41.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:31.782" grpc.time_ms=10.366 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.003404] gitaly[891]: time="2026-09-13T10:52:31.825Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=5 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:52:41.023" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:31.804" grpc.time_ms=19.818 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.047592] gitaly[891]: time="2026-09-13T10:52:31.922Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:52:41.125" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:31.905" grpc.time_ms=16.756 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.103874] gitaly[891]: time="2026-09-13T10:52:31.979Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:31.978" grpc.time_ms=0.69 pid=891 span.kind=server system=grpc gitlab # [ 427.159239] gitaly[891]: time="2026-09-13T10:52:32.034Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:32.033" grpc.time_ms=0.689 pid=891 span.kind=server system=grpc gitlab # [ 427.178604] gitaly[891]: time="2026-09-13T10:52:32.051Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG diskcache=82ed0821-fc0e-48ed-bef3-6c1bb06a78be grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:52:41.159" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:31.940" pid=891 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 427.196525] gitaly[891]: time="2026-09-13T10:52:32.052Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=112 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4908 command.oublock=8 command.real_time_ms=104 command.spawn_token_fork_ms=1 command.system_time_ms=68 command.user_time_ms=44 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:52:41.159" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:31.940" grpc.time_ms=111.685 pid=891 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 427.215741] gitaly[891]: time="2026-09-13T10:52:32.082Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-13T10:52:42.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:32.081" grpc.time_ms=0.302 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.239301] gitaly[891]: time="2026-09-13T10:52:32.114Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-13T10:52:42.009" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:32.090" grpc.time_ms=23.896 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.421655] gitaly[891]: time="2026-09-13T10:52:32.296Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=312 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-13T10:53:02.081" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:32.281" grpc.time_ms=15.162 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.484542] gitaly[891]: time="2026-09-13T10:52:32.359Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:52:42.278" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:32.358" grpc.time_ms=0.523 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.514389] gitaly[891]: time="2026-09-13T10:52:32.376Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:52:42.283" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:52:32.364" grpc.time_ms=11.667 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.547633] gitaly[891]: time="2026-09-13T10:52:32.422Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:42.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:32.404" grpc.time_ms=17.818 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.585670] gitaly[891]: time="2026-09-13T10:52:32.451Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:42.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:32.434" grpc.time_ms=16.427 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.629165] gitaly[891]: time="2026-09-13T10:52:32.504Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=17 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:53:02.275" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:32.475" grpc.time_ms=28.357 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.06 seconds) gitlab # [ 428.021300] nixbot[3588]: INFO: ::ffff:127.0.0.1:42734 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 428.203529] gitaly[891]: time="2026-09-13T10:52:33.078Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:52:43.006" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:33.076" grpc.time_ms=1.404 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.275866] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.149Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"73d47bb01e85f897a8dd1cf0","created_at":"2026-09-13T10:51:07.988Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-bca2a959a34c4e41","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:51:07.997Z","job_size_bytes":185,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-73d47bb01e85f897a8dd1cf0: done: 1.429122 sec","job_status":"done","queue_duration_s":83.72227,"scheduling_latency_s":83.722298,"gitaly_calls":13,"gitaly_duration_s":0.334079,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.06769599999999999,"redis_read_bytes":2103,"redis_write_bytes":21747,"redis_cache_calls":15,"redis_cache_duration_s":0.006746,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.009734,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002468,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7835,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.006179,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.027739,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.01483,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.307,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.355,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.086,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.95272,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00097688,"exclusive_lock_hold_duration_s":0.03750748,"duration_s":1.429122,"completed_at":"2026-09-13T10:52:33.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.150203,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.353960] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.152Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ab7057b7f98c9da6c00a4d7d","created_at":"2026-09-13T10:51:14.220Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.235Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ab7057b7f98c9da6c00a4d7d: start","job_status":"start","queue_duration_s":78.917515,"scheduling_latency_s":78.917562} gitlab # [ 428.464107] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.338Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"ab7057b7f98c9da6c00a4d7d","created_at":"2026-09-13T10:51:14.220Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.235Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-ab7057b7f98c9da6c00a4d7d: done: 0.185664 sec","job_status":"done","queue_duration_s":78.917515,"scheduling_latency_s":78.917562,"redis_calls":16,"redis_duration_s":0.007042,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003541,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000774,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.134367,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00088382,"duration_s":0.185664,"completed_at":"2026-09-13T10:52:33.338Z","load_balancing_strategy":"primary","db_duration_s":0.036498,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.552391] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.342Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"325657a62a9e2f98cac820cd","created_at":"2026-09-13T10:51:14.236Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.247Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-325657a62a9e2f98cac820cd: start","job_status":"start","queue_duration_s":79.094712,"scheduling_latency_s":79.094756} gitlab # [ 428.807465] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.681Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"325657a62a9e2f98cac820cd","created_at":"2026-09-13T10:51:14.236Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.247Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-325657a62a9e2f98cac820cd: done: 0.339189 sec","job_status":"done","queue_duration_s":79.094712,"scheduling_latency_s":79.094756,"redis_calls":21,"redis_duration_s":0.008571,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003941,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002145,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002485,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.267854,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060312,"duration_s":0.339189,"completed_at":"2026-09-13T10:52:33.681Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05754,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.910881] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:33.684Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"19e07c224ec6b6ca8a4b8c9f","created_at":"2026-09-13T10:51:14.436Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.446Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-19e07c224ec6b6ca8a4b8c9f: start","job_status":"start","queue_duration_s":79.238316,"scheduling_latency_s":79.238341} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 429.062986] gitaly[891]: time="2026-09-13T10:52:33.937Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:33.936" grpc.time_ms=0.924 pid=891 span.kind=server system=grpc gitlab # [ 429.091384] nixbot[3588]: INFO: ::ffff:127.0.0.1:42748 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.07 seconds) gitlab # [ 429.130496] gitaly[891]: time="2026-09-13T10:52:34.005Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:34.004" grpc.time_ms=1.365 pid=891 span.kind=server system=grpc gitlab # [ 429.181388] gitaly[891]: time="2026-09-13T10:52:34.056Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:34.055" grpc.time_ms=0.717 pid=891 span.kind=server system=grpc gitlab # [ 429.204360] gitaly[891]: time="2026-09-13T10:52:34.069Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS diskcache=89bc7d63-0087-40c0-ab17-17c6d276d61f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:03.677" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:33.878" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.222564] gitaly[891]: time="2026-09-13T10:52:34.070Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=203 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7640 command.oublock=0 command.real_time_ms=184 command.spawn_token_fork_ms=1 command.system_time_ms=133 command.user_time_ms=70 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:03.677" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:33.878" grpc.time_ms=191.28 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.418329] gitaly[891]: time="2026-09-13T10:52:34.293Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:04.090" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:34.291" grpc.time_ms=1.872 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.508613] gitaly[891]: time="2026-09-13T10:52:34.383Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D690Q0N3TYSQ0KB7N6H8XS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:04.181" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:34.381" grpc.time_ms=1.635 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.552909] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.426Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"19e07c224ec6b6ca8a4b8c9f","created_at":"2026-09-13T10:51:14.436Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-f7ef69c2ce1a4698","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:51:14.446Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-19e07c224ec6b6ca8a4b8c9f: done: 0.741811 sec","job_status":"done","queue_duration_s":79.238316,"scheduling_latency_s":79.238341,"gitaly_calls":3,"gitaly_duration_s":0.207792,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.040445,"redis_read_bytes":455,"redis_write_bytes":27997,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001599,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.004573,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.0137,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.006052,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.008186,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000847,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000804,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003085,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.097,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.034,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.038,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.410893,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.6474087000000001,"duration_s":0.741811,"completed_at":"2026-09-13T10:52:34.426Z","load_balancing_strategy":"primary","db_duration_s":0.113643,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.649478] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.429Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c 3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"15c41aeff3a70388113d6e56","created_at":"2026-09-13T10:52:29.419Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-df9af029859d45da","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:29.424Z","job_size_bytes":185,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-15c41aeff3a70388113d6e56: start","job_status":"start","queue_duration_s":5.004351,"scheduling_latency_s":5.004387} gitlab # [ 429.667383] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.449Z","message":"changes: \"541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c 3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 refs/heads/master\\n\""} gitlab # [ 429.669938] gitaly[891]: time="2026-09-13T10:52:34.498Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:44.079" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:34.480" grpc.time_ms=18.331 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.683020] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.532Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"32d7b47b6901875a9c9ee3df","created_at":"2026-09-13T10:52:34.528Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"0fa0da9e945d0b8b62841dcf","job_size_bytes":46,"pid":3454,"job_status":"deduplicated","message":"ProjectCacheWorker JID-32d7b47b6901875a9c9ee3df: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 429.697732] gitaly[891]: time="2026-09-13T10:52:34.523Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-13T10:52:44.105" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:34.506" grpc.time_ms=17.48 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.711978] gitaly[891]: time="2026-09-13T10:52:34.579Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-13T10:52:44.160" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:34.561" grpc.time_ms=18.416 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.735113] gitaly[891]: time="2026-09-13T10:52:34.610Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=598 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-13T10:53:04.389" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:34.590" grpc.time_ms=20.241 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.767952] gitaly[891]: time="2026-09-13T10:52:34.643Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-13T10:52:44.231" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-13T10:52:34.631" grpc.time_ms=11.068 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.802670] gitaly[891]: time="2026-09-13T10:52:34.677Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:44.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:34.661" grpc.time_ms=16.195 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.988516] gitaly[891]: time="2026-09-13T10:52:34.863Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-13T10:52:44.080" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:34.861" grpc.time_ms=2.404 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 430.027421] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.900Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c 3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"15c41aeff3a70388113d6e56","created_at":"2026-09-13T10:52:29.419Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-df9af029859d45da","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:29.424Z","job_size_bytes":185,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-15c41aeff3a70388113d6e56: done: 0.471381 sec","job_status":"done","queue_duration_s":5.004351,"scheduling_latency_s":5.004387,"gitaly_calls":7,"gitaly_duration_s":0.134184,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.037089,"redis_read_bytes":501,"redis_write_bytes":11942,"redis_cache_calls":9,"redis_cache_duration_s":0.004506,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001655,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5719,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005974,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.01197,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.012984,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.047,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.047,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.042,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.292503,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0011059,"exclusive_lock_hold_duration_s":0.03273326,"duration_s":0.471381,"completed_at":"2026-09-13T10:52:34.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043817,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.110334] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:34.903Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0fa0da9e945d0b8b62841dcf","created_at":"2026-09-13T10:52:31.828Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:31.852Z","job_size_bytes":46,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0fa0da9e945d0b8b62841dcf: start","job_status":"start","queue_duration_s":3.051597,"scheduling_latency_s":3.051634} gitlab # [ 430.130124] gitaly[891]: time="2026-09-13T10:52:34.983Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-13T16:52:34.972" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:34.973" grpc.time_ms=9.618 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.06 seconds) gitlab # [ 430.142395] gitaly[891]: time="2026-09-13T10:52:35.015Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:53:04.794" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:34.995" grpc.time_ms=19.519 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.158083] nixbot[3588]: INFO: ::ffff:127.0.0.1:42750 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 430.193974] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.068Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"0fa0da9e945d0b8b62841dcf","created_at":"2026-09-13T10:52:31.828Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:31.852Z","job_size_bytes":46,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0fa0da9e945d0b8b62841dcf: done: 0.16437 sec","job_status":"done","queue_duration_s":3.051597,"scheduling_latency_s":3.051634,"gitaly_calls":2,"gitaly_duration_s":0.035928,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.009592,"redis_read_bytes":69,"redis_write_bytes":4885,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001883,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000811,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2113,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002347,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003429,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001122,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097297,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00055608,"duration_s":0.16437,"completed_at":"2026-09-13T10:52:35.068Z","load_balancing_strategy":"primary","db_duration_s":0.018975,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.260357] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.071Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"da555613ebb8888b776e929b","created_at":"2026-09-13T10:52:31.887Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:31.898Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-da555613ebb8888b776e929b: start","job_status":"start","queue_duration_s":3.172643,"scheduling_latency_s":3.172677} gitlab # [ 430.278256] gitaly[891]: time="2026-09-13T10:52:35.152Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG diskcache=9cc977d8-c01a-45de-9a38-7c0c27e4f2ac grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:52:35.114" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.115" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.288375] gitaly[891]: time="2026-09-13T10:52:35.153Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=45 command.spawn_token_fork_ms=4 command.system_time_ms=19 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:52:35.114" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.115" grpc.time_ms=37.467 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.358586] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.232Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"da555613ebb8888b776e929b","created_at":"2026-09-13T10:52:31.887Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:31.898Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-da555613ebb8888b776e929b: done: 0.161643 sec","job_status":"done","queue_duration_s":3.172643,"scheduling_latency_s":3.172677,"gitaly_calls":1,"gitaly_duration_s":0.041404,"redis_calls":15,"redis_duration_s":0.0058649999999999996,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000704,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002434,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000906,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001821,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.023,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.041,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11164032,"duration_s":0.161643,"completed_at":"2026-09-13T10:52:35.232Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028284,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.427249] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.236Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"61c12b5f114f1d2cc072652c","created_at":"2026-09-13T10:52:32.073Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:32.077Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-61c12b5f114f1d2cc072652c: start","job_status":"start","queue_duration_s":3.159217,"scheduling_latency_s":3.159268} gitlab # [ 430.444045] gitaly[891]: time="2026-09-13T10:52:35.317Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=594 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-13T10:53:05.098" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.298" grpc.time_ms=18.244 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.462012] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.336Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"61c12b5f114f1d2cc072652c","created_at":"2026-09-13T10:52:32.073Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:32.077Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-61c12b5f114f1d2cc072652c: done: 0.100127 sec","job_status":"done","queue_duration_s":3.159217,"scheduling_latency_s":3.159268,"gitaly_calls":1,"gitaly_duration_s":0.022216,"redis_calls":16,"redis_duration_s":0.006108,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000514,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002541,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.062114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.100127,"completed_at":"2026-09-13T10:52:35.336Z","load_balancing_strategy":"primary","db_duration_s":0.005518,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.509752] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:35.339Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"de4737d1b8c6cdda3fd87072","created_at":"2026-09-13T10:52:32.887Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:328c7dc0e49d3ad700170ea68db6a9c79a47710f0f99c1f1c4ffd127cf817d09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:32.894Z","job_size_bytes":467,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-de4737d1b8c6cdda3fd87072: start","job_status":"start","queue_duration_s":2.445129,"scheduling_latency_s":2.445167} gitlab # [ 430.528141] gitaly[891]: time="2026-09-13T10:52:35.384Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:05.182" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.382" grpc.time_ms=1.069 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.540232] gitaly[891]: time="2026-09-13T10:52:35.405Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:52:45.007" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:35.387" grpc.time_ms=16.895 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.746260] gitaly[891]: time="2026-09-13T10:52:35.621Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:05.419" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.620" grpc.time_ms=1.279 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.762241] gitaly[891]: time="2026-09-13T10:52:35.634Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=38 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:53:05.429" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:35.629" grpc.time_ms=4.453 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.933652] gitaly[891]: time="2026-09-13T10:52:35.808Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:45.010" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:35.791" grpc.time_ms=17.46 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 431.215696] nixbot[3588]: INFO: ::ffff:127.0.0.1:44870 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.07 seconds) gitlab # [ 431.562727] gitaly[891]: time="2026-09-13T10:52:36.436Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:53:06.009" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:36.409" grpc.time_ms=26.752 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.605951] gitaly[891]: time="2026-09-13T10:52:36.462Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D68THKTEWRA0D4ZHZX5GDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:53:06.039" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:36.440" grpc.time_ms=21.456 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.632957] sidekiq-cluster[3454]: {"severity":"WARN","time":"2026-09-13T10:52:36.499Z","class":"CreatePipelineWorker","correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c","ref":"refs/heads/master"},"retry":0} gitlab # [ 431.644154] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"de4737d1b8c6cdda3fd87072","created_at":"2026-09-13T10:52:32.887Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:328c7dc0e49d3ad700170ea68db6a9c79a47710f0f99c1f1c4ffd127cf817d09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:32.894Z","job_size_bytes":467,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-de4737d1b8c6cdda3fd87072: done: 1.171419 sec","job_status":"done","queue_duration_s":2.445129,"scheduling_latency_s":2.445167,"gitaly_calls":7,"gitaly_duration_s":0.113011,"redis_calls":51,"redis_duration_s":0.014024,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001336,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005712,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00206,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003515,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001401,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.979391,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049548,"duration_s":1.171419,"completed_at":"2026-09-13T10:52:36.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.085288,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 431.708934] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.513Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e39251a6-cac1-4603-8fd2-0edcdc7f73ff\"}"],"class":"WebHookWorker","jid":"f62631cdf496f12fd34f31ee","created_at":"2026-09-13T10:52:33.039Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ecfb238069a56b1d68f6cee2c5f5a54c930c8c681500eb20c94d0a4a0c4c2411","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:33.045Z","job_size_bytes":1751,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f62631cdf496f12fd34f31ee: start","job_status":"start","queue_duration_s":3.467788,"scheduling_latency_s":3.467817} gitlab # [ 431.804193] nixbot[3588]: INFO: ::ffff:127.0.0.1:44882 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 431.841964] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.715Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e39251a6-cac1-4603-8fd2-0edcdc7f73ff\"}"],"class":"WebHookWorker","jid":"f62631cdf496f12fd34f31ee","created_at":"2026-09-13T10:52:33.039Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-4ac9368b4cb445b4","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:ecfb238069a56b1d68f6cee2c5f5a54c930c8c681500eb20c94d0a4a0c4c2411","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:33.045Z","job_size_bytes":1751,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-f62631cdf496f12fd34f31ee: done: 0.202652 sec","job_status":"done","queue_duration_s":3.467788,"scheduling_latency_s":3.467817,"redis_calls":19,"redis_duration_s":0.007000999999999999,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001996,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000663,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003194,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001148,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.07567081999997072,"cpu_s":0.106047,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00110644,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.202652,"completed_at":"2026-09-13T10:52:36.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.939877] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.719Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3c9360a981deb049e33b72d8","created_at":"2026-09-13T10:52:33.854Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:33.861Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3c9360a981deb049e33b72d8: start","job_status":"start","queue_duration_s":2.85745,"scheduling_latency_s":2.857486} gitlab # [ 431.976614] gitaly[891]: time="2026-09-13T10:52:36.851Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BHZWF7GDRA19Y4B1WZJ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:36.836" grpc.time_ms=15.356 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.998992] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.853Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3c9360a981deb049e33b72d8","created_at":"2026-09-13T10:52:33.854Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:33.861Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3c9360a981deb049e33b72d8: done: 0.134829 sec","job_status":"done","queue_duration_s":2.85745,"scheduling_latency_s":2.857486,"redis_calls":10,"redis_duration_s":0.002878,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000765,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00059,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100138,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054922,"duration_s":0.134829,"completed_at":"2026-09-13T10:52:36.853Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027315,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.056924] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.855Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c475d2a43bac9fa3468dbda3","created_at":"2026-09-13T10:52:34.081Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.093Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c475d2a43bac9fa3468dbda3: start","job_status":"start","queue_duration_s":2.762326,"scheduling_latency_s":2.762352} gitlab # [ 432.074347] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:36 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 127 gitlab # [ 432.076481] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.900Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c475d2a43bac9fa3468dbda3","created_at":"2026-09-13T10:52:34.081Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.093Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c475d2a43bac9fa3468dbda3: done: 0.044731 sec","job_status":"done","queue_duration_s":2.762326,"scheduling_latency_s":2.762352,"redis_calls":10,"redis_duration_s":0.002002,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001238,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000384,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002866,"duration_s":0.044731,"completed_at":"2026-09-13T10:52:36.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005977,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.120474] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.902Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e90d988428da0b7e1ba02a57","created_at":"2026-09-13T10:52:34.121Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.131Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e90d988428da0b7e1ba02a57: start","job_status":"start","queue_duration_s":2.770945,"scheduling_latency_s":2.770965} gitlab # [ 432.136375] gitaly[891]: time="2026-09-13T10:52:36.973Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ3YJFXVQJBJ8DRVDZE9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:36.957" pid=891 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 432.146503] gitlab-workhorse[815]: time="2026-09-13T10:52:36Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 432.148355] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:36 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 112 gitlab # [ 432.150241] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.941Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"e90d988428da0b7e1ba02a57","created_at":"2026-09-13T10:52:34.121Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.131Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-e90d988428da0b7e1ba02a57: done: 0.039816 sec","job_status":"done","queue_duration_s":2.770945,"scheduling_latency_s":2.770965,"redis_calls":10,"redis_duration_s":0.002173,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001414,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045126,"duration_s":0.039816,"completed_at":"2026-09-13T10:52:36.941Z","load_balancing_strategy":"primary","db_duration_s":0.005978,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.194816] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.943Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"49e4c6480972c07c1d855006","created_at":"2026-09-13T10:52:34.159Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.168Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-49e4c6480972c07c1d855006: start","job_status":"start","queue_duration_s":2.775751,"scheduling_latency_s":2.77577} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 432.210478] gitaly[891]: time="2026-09-13T10:52:36.973Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ3YJFXVQJBJ8DRVDZE9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:36.957" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 432.224285] gitlab-workhorse[815]: time="2026-09-13T10:52:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=342 gitlab # [ 432.226212] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 342 "" "git/2.55.0" 103 gitlab # [ 432.228210] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.987Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"49e4c6480972c07c1d855006","created_at":"2026-09-13T10:52:34.159Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.168Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-49e4c6480972c07c1d855006: done: 0.043387 sec","job_status":"done","queue_duration_s":2.775751,"scheduling_latency_s":2.77577,"redis_calls":13,"redis_duration_s":0.0042,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001367,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001893,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000475,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026412,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002612,"duration_s":0.043387,"completed_at":"2026-09-13T10:52:36.987Z","load_balancing_strategy":"primary","db_duration_s":0.008117,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.06 seconds) gitlab # [ 432.272252] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:36.988Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"50e97118e8658ad9eb2020e3","created_at":"2026-09-13T10:52:34.169Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.178Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-50e97118e8658ad9eb2020e3: start","job_status":"start","queue_duration_s":2.810289,"scheduling_latency_s":2.810307} gitlab # [ 432.288790] gitaly[891]: time="2026-09-13T10:52:36.974Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=0 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ3YJFXVQJBJ8DRVDZE9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:36.957" grpc.time_ms=16.942 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 432.303352] nixbot[3588]: INFO: ::ffff:127.0.0.1:44898 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 432.305327] nixbot[3588]: {"ts": "2026-09-13T10:52:37.149Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 432.307679] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.027Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"50e97118e8658ad9eb2020e3","created_at":"2026-09-13T10:52:34.169Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.178Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-50e97118e8658ad9eb2020e3: done: 0.038654 sec","job_status":"done","queue_duration_s":2.810289,"scheduling_latency_s":2.810307,"redis_calls":10,"redis_duration_s":0.002942,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000541,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025194,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062842,"duration_s":0.038654,"completed_at":"2026-09-13T10:52:37.027Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.352309] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.029Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8e43a5da333e2535de88d668","created_at":"2026-09-13T10:52:34.179Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.189Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8e43a5da333e2535de88d668: start","job_status":"start","queue_duration_s":2.840667,"scheduling_latency_s":2.840686} gitlab # [ 432.369503] gitaly[891]: time="2026-09-13T10:52:37.081Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ7KEWH4WKTW9GJRFFA8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:37.056" pid=891 remote_ip="2001:db8:1::1" request_sha=1e08b2c04d3f85f3ff437b58a8de0cb3199d571c response_bytes=342 span.kind=server system=grpc gitlab # [ 432.379641] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.069Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"8e43a5da333e2535de88d668","created_at":"2026-09-13T10:52:34.179Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.189Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-8e43a5da333e2535de88d668: done: 0.03936 sec","job_status":"done","queue_duration_s":2.840667,"scheduling_latency_s":2.840686,"redis_calls":11,"redis_duration_s":0.0031739999999999997,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002052,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000556,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040204,"duration_s":0.03936,"completed_at":"2026-09-13T10:52:37.068Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009148,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.423491] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.070Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"065d866e6bd2015f03e6a3c4","created_at":"2026-09-13T10:52:34.194Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.201Z","job_size_bytes":66,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-065d866e6bd2015f03e6a3c4: start","job_status":"start","queue_duration_s":2.869045,"scheduling_latency_s":2.869064} gitlab # [ 432.440994] gitaly[891]: time="2026-09-13T10:52:37.082Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1126 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ7KEWH4WKTW9GJRFFA8 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:37.056" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 432.454697] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.111Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"065d866e6bd2015f03e6a3c4","created_at":"2026-09-13T10:52:34.194Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.201Z","job_size_bytes":66,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-065d866e6bd2015f03e6a3c4: done: 0.040643 sec","job_status":"done","queue_duration_s":2.869045,"scheduling_latency_s":2.869064,"redis_calls":10,"redis_duration_s":0.003409,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001069,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001444,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02305,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030248,"duration_s":0.040643,"completed_at":"2026-09-13T10:52:37.111Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007591,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.499636] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.115Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2376df34c94f8ae3d49e85b2","created_at":"2026-09-13T10:52:34.203Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.210Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2376df34c94f8ae3d49e85b2: start","job_status":"start","queue_duration_s":2.904527,"scheduling_latency_s":2.904568} gitlab # [ 432.515811] gitaly[891]: time="2026-09-13T10:52:37.082Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1126 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJ7KEWH4WKTW9GJRFFA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:37.056" grpc.time_ms=25.504 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 432.530652] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.182Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"2376df34c94f8ae3d49e85b2","created_at":"2026-09-13T10:52:34.203Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.210Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2376df34c94f8ae3d49e85b2: done: 0.066625 sec","job_status":"done","queue_duration_s":2.904527,"scheduling_latency_s":2.904568,"redis_calls":10,"redis_duration_s":0.0066820000000000004,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000912,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005152,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057374,"duration_s":0.066625,"completed_at":"2026-09-13T10:52:37.181Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008128,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.576034] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.183Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a7bfcd457ed3686a315713da","created_at":"2026-09-13T10:52:34.219Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.225Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a7bfcd457ed3686a315713da: start","job_status":"start","queue_duration_s":2.958473,"scheduling_latency_s":2.958492} gitlab # [ 432.592734] gitaly[891]: time="2026-09-13T10:52:37.235Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:07.034" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.234" grpc.time_ms=0.711 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.612289] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.250Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a7bfcd457ed3686a315713da","created_at":"2026-09-13T10:52:34.219Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.225Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a7bfcd457ed3686a315713da: done: 0.067154 sec","job_status":"done","queue_duration_s":2.958473,"scheduling_latency_s":2.958492,"redis_calls":10,"redis_duration_s":0.002994,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000569,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.045755,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00111336,"duration_s":0.067154,"completed_at":"2026-09-13T10:52:37.250Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01244,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.656674] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.255Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3107b295d3d6889a6a9ca93a","created_at":"2026-09-13T10:52:34.225Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.232Z","job_size_bytes":6,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3107b295d3d6889a6a9ca93a: start","job_status":"start","queue_duration_s":3.022371,"scheduling_latency_s":3.022413} gitlab # [ 432.672589] gitaly[891]: time="2026-09-13T10:52:37.405Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:47.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:37.394" grpc.time_ms=10.503 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.686070] nixbot[3588]: {"ts": "2026-09-13T10:52:37.533Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 432.688868] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 367 gitlab # [ 432.691404] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.303Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"3107b295d3d6889a6a9ca93a","created_at":"2026-09-13T10:52:34.225Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.232Z","job_size_bytes":6,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-3107b295d3d6889a6a9ca93a: done: 0.048443 sec","job_status":"done","queue_duration_s":3.022371,"scheduling_latency_s":3.022413,"redis_calls":10,"redis_duration_s":0.002703,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000598,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02986,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004668,"duration_s":0.048443,"completed_at":"2026-09-13T10:52:37.303Z","load_balancing_strategy":"primary","db_duration_s":0.004821,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.735293] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.305Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d2495d9d642ff67efb0ebb45","created_at":"2026-09-13T10:52:34.235Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.245Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d2495d9d642ff67efb0ebb45: start","job_status":"start","queue_duration_s":3.060064,"scheduling_latency_s":3.060097} gitlab # [ 432.750628] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.517Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"d2495d9d642ff67efb0ebb45","created_at":"2026-09-13T10:52:34.235Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.245Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-d2495d9d642ff67efb0ebb45: done: 0.212043 sec","job_status":"done","queue_duration_s":3.060064,"scheduling_latency_s":3.060097,"redis_calls":10,"redis_duration_s":0.006031,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041924,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0010531,"duration_s":0.212043,"completed_at":"2026-09-13T10:52:37.517Z","load_balancing_strategy":"primary","db_duration_s":0.015071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.794355] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.519Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a000c9dd300c745874b3e742","created_at":"2026-09-13T10:52:34.258Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.268Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a000c9dd300c745874b3e742: start","job_status":"start","queue_duration_s":3.251462,"scheduling_latency_s":3.251489} gitlab # [ 432.810226] gitaly[891]: time="2026-09-13T10:52:37.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJS4JF6DMA4EZ82S0AAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:07.095" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.596" grpc.time_ms=1.185 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.822683] gitaly[891]: time="2026-09-13T10:52:37.642Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M diskcache=08935b48-40a1-49dc-9709-54b2fe9025fb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:52:37.627" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.628" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.832572] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.604Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a000c9dd300c745874b3e742","created_at":"2026-09-13T10:52:34.258Z","trace_propagation_headers":{"sentry-trace":"fbea17c50c254cd2a866a03a01144bae-63648366749642d5","baggage":"sentry-trace_id=fbea17c50c254cd2a866a03a01144bae,sentry-release=Unknown"},"correlation_id":"01M2D690Q0N3TYSQ0KB7N6H8XS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.268Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a000c9dd300c745874b3e742: done: 0.084958 sec","job_status":"done","queue_duration_s":3.251462,"scheduling_latency_s":3.251489,"redis_calls":14,"redis_duration_s":0.008341,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003594,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002929,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001818,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048832,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054948,"duration_s":0.084958,"completed_at":"2026-09-13T10:52:37.604Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.023268,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.875087] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.606Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7985c380852d8aac85db92e5","created_at":"2026-09-13T10:52:34.547Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"da555613ebb8888b776e929b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.553Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7985c380852d8aac85db92e5: start","job_status":"start","queue_duration_s":3.052989,"scheduling_latency_s":3.053012} gitlab # [ 432.891600] nixbot[3588]: {"ts": "2026-09-13T10:52:37.700Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 432.894435] gitaly[891]: time="2026-09-13T10:52:37.644Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=289 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-13T16:52:37.627" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.628" grpc.time_ms=14.279 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.907410] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 159 gitlab # [ 432.909970] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.671Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"7985c380852d8aac85db92e5","created_at":"2026-09-13T10:52:34.547Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"da555613ebb8888b776e929b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.553Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-7985c380852d8aac85db92e5: done: 0.064828 sec","job_status":"done","queue_duration_s":3.052989,"scheduling_latency_s":3.053012,"gitaly_calls":1,"gitaly_duration_s":0.017659,"redis_calls":12,"redis_duration_s":0.006366,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002579,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03998486,"duration_s":0.064828,"completed_at":"2026-09-13T10:52:37.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010474,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.954260] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:37.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2e9ddab98e2ec5ee5ebd97eb","created_at":"2026-09-13T10:52:34.791Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f33d361dfa9f6ab4f5d991a23659721dc6b220af22cd876e273cd09cc8eeecf2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.798Z","job_size_bytes":467,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2e9ddab98e2ec5ee5ebd97eb: start","job_status":"start","queue_duration_s":2.876433,"scheduling_latency_s":2.876485} gitlab # [ 432.970930] gitaly[891]: time="2026-09-13T10:52:37.651Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJS4JF6DMA4EZ82S0AAG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:47.148" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:37.639" grpc.time_ms=12.561 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.985281] gitaly[891]: time="2026-09-13T10:52:37.722Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:07.319" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.720" grpc.time_ms=1.547 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.997358] gitaly[891]: time="2026-09-13T10:52:37.736Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:52:47.005" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:37.725" grpc.time_ms=10.978 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.010257] gitaly[891]: time="2026-09-13T10:52:37.769Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJYAB31F2JS1PP8ESS1S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:07.268" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.768" grpc.time_ms=0.923 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.022650] gitaly[891]: time="2026-09-13T10:52:37.796Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:07.392" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.793" grpc.time_ms=1.657 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.035206] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 189 gitlab # [ 433.037831] nixbot[3588]: {"ts": "2026-09-13T10:52:37.897Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c \"HTTP/1.1 201 Created\""} gitlab # [ 433.040523] gitaly[891]: time="2026-09-13T10:52:37.808Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=286 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-13T10:53:07.401" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:37.801" grpc.time_ms=6.104 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.055985] gitaly[891]: time="2026-09-13T10:52:37.833Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJYAB31F2JS1PP8ESS1S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:47.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:37.812" grpc.time_ms=21.45 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.069118] gitaly[891]: time="2026-09-13T10:52:37.867Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:47.070" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:37.852" grpc.time_ms=14.596 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.176840] gitaly[891]: time="2026-09-13T10:52:38.051Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=15 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BK619GX8X2SZMAVB22BR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.032" grpc.time_ms=19.198 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 433.211603] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 98 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 433.276975] gitaly[891]: time="2026-09-13T10:52:38.152Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BK9625A646WWPKA9YV4E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.132" pid=891 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 433.304510] gitaly[891]: time="2026-09-13T10:52:38.152Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BK9625A646WWPKA9YV4E grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.132" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 433.331627] gitlab-workhorse[815]: time="2026-09-13T10:52:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 433.335551] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 98 gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.08 seconds) gitlab # [ 433.339699] gitaly[891]: time="2026-09-13T10:52:38.152Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BK9625A646WWPKA9YV4E grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.132" grpc.time_ms=20.245 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 433.364151] nixbot[3588]: INFO: ::ffff:127.0.0.1:44914 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 433.574744] gitaly[891]: time="2026-09-13T10:52:38.449Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=25 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-13T10:53:08.020" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:38.420" grpc.time_ms=28.835 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.621343] gitaly[891]: time="2026-09-13T10:52:38.472Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6B9YDRZYE9SZH00W8EG0M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-13T10:53:08.055" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:38.455" grpc.time_ms=17.233 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.648811] sidekiq-cluster[3454]: {"severity":"WARN","time":"2026-09-13T10:52:38.490Z","class":"CreatePipelineWorker","correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"541e2c75d6ab638dd6f7b1bc34a18b6b0d519f2c","after":"3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3","ref":"refs/heads/master"},"retry":0} gitlab # [ 433.660653] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.497Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2e9ddab98e2ec5ee5ebd97eb","created_at":"2026-09-13T10:52:34.791Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f33d361dfa9f6ab4f5d991a23659721dc6b220af22cd876e273cd09cc8eeecf2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.798Z","job_size_bytes":467,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2e9ddab98e2ec5ee5ebd97eb: done: 0.823337 sec","job_status":"done","queue_duration_s":2.876433,"scheduling_latency_s":2.876485,"gitaly_calls":7,"gitaly_duration_s":0.108924,"redis_calls":33,"redis_duration_s":0.010822,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000747,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.007136,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001322,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.668708,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.823337,"completed_at":"2026-09-13T10:52:38.497Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041952,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 433.726438] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.500Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"40d49e20-8a88-4d95-b81c-4fc79db92c20\"}"],"class":"WebHookWorker","jid":"a206da0e63d5e0448c66d689","created_at":"2026-09-13T10:52:34.829Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdd05b54cf06fd5e84a7703401190c56b36d2e91c0c4a7454419372bf4adc6ab","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.835Z","job_size_bytes":1749,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a206da0e63d5e0448c66d689: start","job_status":"start","queue_duration_s":3.665006,"scheduling_latency_s":3.665037} gitlab # [ 433.745248] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.559Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"40d49e20-8a88-4d95-b81c-4fc79db92c20\"}"],"class":"WebHookWorker","jid":"a206da0e63d5e0448c66d689","created_at":"2026-09-13T10:52:34.829Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-0a34cc615c9b4282","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdd05b54cf06fd5e84a7703401190c56b36d2e91c0c4a7454419372bf4adc6ab","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:34.835Z","job_size_bytes":1749,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-a206da0e63d5e0448c66d689: done: 0.058751 sec","job_status":"done","queue_duration_s":3.665006,"scheduling_latency_s":3.665037,"redis_calls":14,"redis_duration_s":0.004092,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.00053,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00238,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001182,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.009563520000028802,"cpu_s":0.036174,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.058751,"completed_at":"2026-09-13T10:52:38.558Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.792143] nixbot[3588]: INFO: ::ffff:127.0.0.1:44916 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 433.793372] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.562Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c2811bc02b1564f4b48853de","created_at":"2026-09-13T10:52:35.031Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-6340f10ff14d445b","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:35.042Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c2811bc02b1564f4b48853de: start","job_status":"start","queue_duration_s":3.520012,"scheduling_latency_s":3.520062} gitlab # [ 433.810352] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.672Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c2811bc02b1564f4b48853de","created_at":"2026-09-13T10:52:35.031Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-6340f10ff14d445b","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:35.042Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c2811bc02b1564f4b48853de: done: 0.110043 sec","job_status":"done","queue_duration_s":3.520012,"scheduling_latency_s":3.520062,"redis_calls":19,"redis_duration_s":0.0066300000000000005,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001427,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000813,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003173,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001217,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.07455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0145755,"duration_s":0.110043,"completed_at":"2026-09-13T10:52:38.672Z","load_balancing_strategy":"primary","db_duration_s":0.015704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.854799] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.676Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f62631cdf496f12fd34f31ee"],"class":"WebHooks::LogExecutionWorker","jid":"3b936f9be143f120572106f1","created_at":"2026-09-13T10:52:36.683Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-525e2f5555fd4e2a","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6bfda8e6c3bcd785632c83957d8cda98fd12d0084421118aa7c751b6ddd3b99a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:36.695Z","job_size_bytes":2593,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3b936f9be143f120572106f1: start","job_status":"start","queue_duration_s":1.980702,"scheduling_latency_s":1.980753} gitlab # [ 433.871791] gitaly[891]: time="2026-09-13T10:52:38.686Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BKSS80J90V7Y116KDF5N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.671" grpc.time_ms=14.359 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 433.885078] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 101 gitlab # [ 433.891575] gitaly[891]: time="2026-09-13T10:52:38.766Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BKX1JRYCYEZSPD0Z1RRP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.751" pid=891 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 433.901912] gitaly[891]: time="2026-09-13T10:52:38.767Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BKX1JRYCYEZSPD0Z1RRP grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.751" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 433.915430] gitlab-workhorse[815]: time="2026-09-13T10:52:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 433.917353] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 81 gitlab # [ 433.919207] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.786Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","f62631cdf496f12fd34f31ee"],"class":"WebHooks::LogExecutionWorker","jid":"3b936f9be143f120572106f1","created_at":"2026-09-13T10:52:36.683Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-525e2f5555fd4e2a","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6bfda8e6c3bcd785632c83957d8cda98fd12d0084421118aa7c751b6ddd3b99a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:36.695Z","job_size_bytes":2593,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3b936f9be143f120572106f1: done: 0.109678 sec","job_status":"done","queue_duration_s":1.980702,"scheduling_latency_s":1.980753,"redis_calls":10,"redis_duration_s":0.004342,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000819,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002571,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.073295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00085386,"duration_s":0.109678,"completed_at":"2026-09-13T10:52:38.786Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018546,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.963524] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:38.789Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9df7d4ad1f8b7c7a565f4446","created_at":"2026-09-13T10:52:37.505Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-730ab68261bf4769","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:37.509Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9df7d4ad1f8b7c7a565f4446: start","job_status":"start","queue_duration_s":1.280527,"scheduling_latency_s":1.28057} gitlab # [ 433.979439] gitaly[891]: time="2026-09-13T10:52:38.767Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BKX1JRYCYEZSPD0Z1RRP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.751" grpc.time_ms=15.884 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 434.034442] gitaly[891]: time="2026-09-13T10:52:38.909Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM1M5085R6DV57JPK0N5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.893" grpc.time_ms=15.673 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 434.051714] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 73 gitlab # [ 434.108794] gitaly[891]: time="2026-09-13T10:52:38.984Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:38.975" grpc.time_ms=8.064 pid=891 span.kind=server system=grpc gitlab # [ 434.130717] gitaly[891]: time="2026-09-13T10:52:39.005Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM41H2HN1W16JJ9N2GYT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.971" pid=891 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 434.140975] gitaly[891]: time="2026-09-13T10:52:39.006Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=2 command.system_time_ms=21 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM41H2HN1W16JJ9N2GYT grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.971" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 434.154672] gitlab-workhorse[815]: time="2026-09-13T10:52:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 434.156720] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 93 gitlab # [ 434.158922] gitaly[891]: time="2026-09-13T10:52:39.006Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=31 command.spawn_token_fork_ms=2 command.system_time_ms=21 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM41H2HN1W16JJ9N2GYT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:38.971" grpc.time_ms=34.477 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 434.173633] gitaly[891]: time="2026-09-13T10:52:39.022Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:39.021" grpc.time_ms=0.717 pid=891 span.kind=server system=grpc gitlab # [ 434.183076] gitaly[891]: time="2026-09-13T10:52:39.054Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:39.054" grpc.time_ms=0.56 pid=891 span.kind=server system=grpc gitlab # [ 434.193191] gitaly[891]: time="2026-09-13T10:52:39.064Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE diskcache=506b5900-cf38-4593-afaf-f37581329cbb grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:08.515" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:38.916" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.202725] gitaly[891]: time="2026-09-13T10:52:39.064Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=148 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7329 command.oublock=0 command.real_time_ms=142 command.spawn_token_fork_ms=1 command.system_time_ms=90 command.user_time_ms=58 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:08.515" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:38.916" grpc.time_ms=148.179 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.265639] gitaly[891]: time="2026-09-13T10:52:39.140Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:09.039" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:39.140" grpc.time_ms=0.827 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.278599] gitaly[891]: time="2026-09-13T10:52:39.142Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:09.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:39.141" grpc.time_ms=0.839 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.300798] gitaly[891]: time="2026-09-13T10:52:39.176Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=10 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-13T10:53:09.062" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:39.163" grpc.time_ms=12.82 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.315119] gitaly[891]: time="2026-09-13T10:52:39.187Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BJDB846B29EXYAVN44QE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:09.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:39.186" grpc.time_ms=0.79 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.327884] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.200Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9df7d4ad1f8b7c7a565f4446","created_at":"2026-09-13T10:52:37.505Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-730ab68261bf4769","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:37.509Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3454,"job_status":"rescheduled","message":"PipelineProcessWorker JID-9df7d4ad1f8b7c7a565f4446: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3' gitlab # [ 434.343173] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.213Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"9df7d4ad1f8b7c7a565f4446","created_at":"2026-09-13T10:52:37.505Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-730ab68261bf4769","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:37.509Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9df7d4ad1f8b7c7a565f4446: done: 0.423291 sec","job_status":"done","queue_duration_s":1.280527,"scheduling_latency_s":1.28057,"gitaly_calls":3,"gitaly_duration_s":0.159034,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.019213,"redis_read_bytes":148,"redis_write_bytes":18018,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.00055,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002635,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004789,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12317,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.006954,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000277,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001036,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002422,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.049,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.204425,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3485175,"duration_s":0.423291,"completed_at":"2026-09-13T10:52:39.212Z","load_balancing_strategy":"primary","db_duration_s":0.05025,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 434.393884] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.216Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a206da0e63d5e0448c66d689"],"class":"WebHooks::LogExecutionWorker","jid":"3ce93a01ddff3abe9d7c2577","created_at":"2026-09-13T10:52:38.539Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-2cf6751dd48945a2","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:44fcd311827dfe567842cb3346d40034e986c3c433686cb06a88c0454504c796","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.544Z","job_size_bytes":2592,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3ce93a01ddff3abe9d7c2577: start","job_status":"start","queue_duration_s":0.671826,"scheduling_latency_s":0.671867} gitlab # [ 434.411178] gitaly[891]: time="2026-09-13T10:52:39.207Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:49.184" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:39.195" grpc.time_ms=12.383 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', in 0.10 seconds) (finished: subtest: push delivers the webhook, in 87.84 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 434.428090] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.257Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","a206da0e63d5e0448c66d689"],"class":"WebHooks::LogExecutionWorker","jid":"3ce93a01ddff3abe9d7c2577","created_at":"2026-09-13T10:52:38.539Z","trace_propagation_headers":{"sentry-trace":"c3faae110bdc4b499522a4fd99b3efe7-2cf6751dd48945a2","baggage":"sentry-trace_id=c3faae110bdc4b499522a4fd99b3efe7,sentry-release=Unknown"},"correlation_id":"01M2D6B9YDRZYE9SZH00W8EG0M","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:44fcd311827dfe567842cb3346d40034e986c3c433686cb06a88c0454504c796","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.544Z","job_size_bytes":2592,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3ce93a01ddff3abe9d7c2577: done: 0.041143 sec","job_status":"done","queue_duration_s":0.671826,"scheduling_latency_s":0.671867,"redis_calls":7,"redis_duration_s":0.002163,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.041143,"completed_at":"2026-09-13T10:52:39.257Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005118,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.474878] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.259Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ff8db6881893e28a00be0aae","created_at":"2026-09-13T10:52:38.646Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-977ff7fafb634333","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.656Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ff8db6881893e28a00be0aae: start","job_status":"start","queue_duration_s":0.602992,"scheduling_latency_s":0.603028} gitlab # [ 434.492919] gitaly[891]: time="2026-09-13T10:52:39.237Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-13T10:52:49.225" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:39.236" grpc.time_ms=0.633 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.504472] nixbot[3588]: INFO: ::ffff:127.0.0.1:44922 - "GET /api/repos/gitlab/root/test-flake/builds?commit=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 200 OK gitlab # [ 434.506399] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.374Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"ff8db6881893e28a00be0aae","created_at":"2026-09-13T10:52:38.646Z","trace_propagation_headers":{"sentry-trace":"0e65cc24d7474da998e510fee54a1cac-977ff7fafb634333","baggage":"sentry-trace_id=0e65cc24d7474da998e510fee54a1cac,sentry-release=Unknown"},"correlation_id":"01M2D68THKTEWRA0D4ZHZX5GDG","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.656Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-ff8db6881893e28a00be0aae: done: 0.115075 sec","job_status":"done","queue_duration_s":0.602992,"scheduling_latency_s":0.603028,"redis_calls":11,"redis_duration_s":0.002102,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000383,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.013,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.080342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027408,"duration_s":0.115075,"completed_at":"2026-09-13T10:52:39.374Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026723,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.551164] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ace156186262275eac0bb5ab","created_at":"2026-09-13T10:52:38.903Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.906Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ace156186262275eac0bb5ab: start","job_status":"start","queue_duration_s":0.46955,"scheduling_latency_s":0.469578} gitlab # [ 434.567740] gitaly[891]: time="2026-09-13T10:52:39.255Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-13T10:52:49.232" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:39.242" grpc.time_ms=12.847 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.581365] gitaly[891]: time="2026-09-13T10:52:39.305Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:39.304" grpc.time_ms=0.497 pid=891 span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.16 seconds) gitlab # [ 434.590899] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.423Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"ace156186262275eac0bb5ab","created_at":"2026-09-13T10:52:38.903Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:38.906Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-ace156186262275eac0bb5ab: done: 0.047048 sec","job_status":"done","queue_duration_s":0.46955,"scheduling_latency_s":0.469578,"redis_calls":7,"redis_duration_s":0.001056,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000735,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000321,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0339,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047048,"completed_at":"2026-09-13T10:52:39.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009325,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 434.635331] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.424Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"300931a7c3237e5fbbb9dfcc","created_at":"2026-09-13T10:52:39.071Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.074Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-300931a7c3237e5fbbb9dfcc: start","job_status":"start","queue_duration_s":0.349787,"scheduling_latency_s":0.349817} gitlab # [ 434.652590] gitaly[891]: time="2026-09-13T10:52:39.362Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:39.359" grpc.time_ms=2.465 pid=891 span.kind=server system=grpc gitlab # [ 434.662196] gitaly[891]: time="2026-09-13T10:52:39.370Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B diskcache=58ff6d56-00cd-4a0e-b1de-34e5ec559ea3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:52:49.248" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:39.258" pid=891 ref=refs/keep-around/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 remote_ip="2001:db8:1::1" revision=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 span.kind=server system=grpc username=root gitlab # [ 434.674116] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"300931a7c3237e5fbbb9dfcc","created_at":"2026-09-13T10:52:39.071Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.074Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-300931a7c3237e5fbbb9dfcc: done: 0.031982 sec","job_status":"done","queue_duration_s":0.349787,"scheduling_latency_s":0.349817,"redis_calls":7,"redis_duration_s":0.00103,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000741,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031982,"completed_at":"2026-09-13T10:52:39.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004439,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.716916] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.458Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2c5d06e1e53cabef4ecc7939","created_at":"2026-09-13T10:52:39.085Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.089Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2c5d06e1e53cabef4ecc7939: start","job_status":"start","queue_duration_s":0.368074,"scheduling_latency_s":0.368104} gitlab # [ 434.732726] nixbot[3588]: {"ts": "2026-09-13T10:52:39.500Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 \"HTTP/1.1 201 Created\""} gitlab # [ 434.735416] nixbot[3588]: {"ts": "2026-09-13T10:52:39.511Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-3c4e47efc34b-5"} gitlab # [ 434.738146] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:39 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 87 gitlab # [ 434.740766] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 431 gitlab # [ 434.743286] gitaly[891]: time="2026-09-13T10:52:39.370Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=71 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4996 command.oublock=8 command.real_time_ms=107 command.spawn_token_fork_ms=0 command.system_time_ms=27 command.user_time_ms=44 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-13T10:52:49.248" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-13T10:52:39.258" grpc.time_ms=111.927 pid=891 ref=refs/keep-around/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 remote_ip="2001:db8:1::1" revision=3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 span.kind=server system=grpc username=root gitlab # [ 434.758191] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.490Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"2c5d06e1e53cabef4ecc7939","created_at":"2026-09-13T10:52:39.085Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.089Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-2c5d06e1e53cabef4ecc7939: done: 0.03214 sec","job_status":"done","queue_duration_s":0.368074,"scheduling_latency_s":0.368104,"redis_calls":7,"redis_duration_s":0.0010379999999999999,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000289,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03214,"completed_at":"2026-09-13T10:52:39.490Z","load_balancing_strategy":"primary","db_duration_s":0.004605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.800341] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.491Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0eb034715635ef807fb1595a","created_at":"2026-09-13T10:52:39.100Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.104Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0eb034715635ef807fb1595a: start","job_status":"start","queue_duration_s":0.387465,"scheduling_latency_s":0.387494} gitlab # [ 434.815524] gitaly[891]: time="2026-09-13T10:52:39.375Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:09.174" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:39.374" grpc.time_ms=1.105 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.829324] nixbot[3588]: {"ts": "2026-09-13T10:52:39.677Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-3c4e47efc34b-5", "argv0": "bwrap"} gitlab # [ 434.831669] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.523Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"0eb034715635ef807fb1595a","created_at":"2026-09-13T10:52:39.100Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.104Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-0eb034715635ef807fb1595a: done: 0.032318 sec","job_status":"done","queue_duration_s":0.387465,"scheduling_latency_s":0.387494,"redis_calls":10,"redis_duration_s":0.001496,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000776,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000426,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000294,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023634,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032318,"completed_at":"2026-09-13T10:52:39.523Z","load_balancing_strategy":"primary","db_duration_s":0.003911,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.875635] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.525Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"46e11718b648c3bc1d7d456b","created_at":"2026-09-13T10:52:39.104Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.107Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-46e11718b648c3bc1d7d456b: start","job_status":"start","queue_duration_s":0.41826,"scheduling_latency_s":0.418289} gitlab # [ 434.892095] gitaly[891]: time="2026-09-13T10:52:39.444Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BMJQP9MYKJ7ETWV09QS9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:09.243" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:39.443" grpc.time_ms=0.575 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.904544] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.555Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"46e11718b648c3bc1d7d456b","created_at":"2026-09-13T10:52:39.104Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.107Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-46e11718b648c3bc1d7d456b: done: 0.030029 sec","job_status":"done","queue_duration_s":0.41826,"scheduling_latency_s":0.418289,"redis_calls":7,"redis_duration_s":0.001502,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001177,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021357,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030029,"completed_at":"2026-09-13T10:52:39.555Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003509,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.946921] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0a226d2baec72d3686d2ac0c","created_at":"2026-09-13T10:52:39.108Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.111Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0a226d2baec72d3686d2ac0c: start","job_status":"start","queue_duration_s":0.445623,"scheduling_latency_s":0.44565} gitlab # [ 434.963108] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.584Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"0a226d2baec72d3686d2ac0c","created_at":"2026-09-13T10:52:39.108Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.111Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-0a226d2baec72d3686d2ac0c: done: 0.027541 sec","job_status":"done","queue_duration_s":0.445623,"scheduling_latency_s":0.44565,"redis_calls":8,"redis_duration_s":0.001572,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000338,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027541,"completed_at":"2026-09-13T10:52:39.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.006225] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.586Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a6c4db5af6cceeed9d28ca49","created_at":"2026-09-13T10:52:39.112Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.114Z","job_size_bytes":66,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a6c4db5af6cceeed9d28ca49: start","job_status":"start","queue_duration_s":0.471916,"scheduling_latency_s":0.47194} gitlab # [ 435.023584] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.616Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"a6c4db5af6cceeed9d28ca49","created_at":"2026-09-13T10:52:39.112Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.114Z","job_size_bytes":66,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-a6c4db5af6cceeed9d28ca49: done: 0.030258 sec","job_status":"done","queue_duration_s":0.471916,"scheduling_latency_s":0.47194,"redis_calls":7,"redis_duration_s":0.001252,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000347,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030258,"completed_at":"2026-09-13T10:52:39.616Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.067701] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.618Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a1d48fec6027bea18881a693","created_at":"2026-09-13T10:52:39.115Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.118Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a1d48fec6027bea18881a693: start","job_status":"start","queue_duration_s":0.49976,"scheduling_latency_s":0.499782} gitlab # [ 435.085046] nix-daemon[3760]: accepted connection from pid 4781, user nixbot gitlab # [ 435.086052] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.648Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"a1d48fec6027bea18881a693","created_at":"2026-09-13T10:52:39.115Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.118Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-a1d48fec6027bea18881a693: done: 0.030763 sec","job_status":"done","queue_duration_s":0.49976,"scheduling_latency_s":0.499782,"redis_calls":7,"redis_duration_s":0.0015199999999999999,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000413,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030763,"completed_at":"2026-09-13T10:52:39.648Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00561,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.128845] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.650Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a55b1b4adc4c064b78c98d76","created_at":"2026-09-13T10:52:39.119Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.122Z","job_size_bytes":6,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a55b1b4adc4c064b78c98d76: start","job_status":"start","queue_duration_s":0.528442,"scheduling_latency_s":0.528461} gitlab # [ 435.144481] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.665Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"a55b1b4adc4c064b78c98d76","created_at":"2026-09-13T10:52:39.119Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.122Z","job_size_bytes":6,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-a55b1b4adc4c064b78c98d76: done: 0.015222 sec","job_status":"done","queue_duration_s":0.528442,"scheduling_latency_s":0.528461,"redis_calls":7,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000429,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009355,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015222,"completed_at":"2026-09-13T10:52:39.665Z","load_balancing_strategy":"primary","db_duration_s":0.00059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.186187] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.667Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1e484040dc0c43643009673b","created_at":"2026-09-13T10:52:39.122Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.125Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1e484040dc0c43643009673b: start","job_status":"start","queue_duration_s":0.541737,"scheduling_latency_s":0.541756} gitlab # [ 435.201328] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.713Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"1e484040dc0c43643009673b","created_at":"2026-09-13T10:52:39.122Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.125Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-1e484040dc0c43643009673b: done: 0.046624 sec","job_status":"done","queue_duration_s":0.541737,"scheduling_latency_s":0.541756,"redis_calls":7,"redis_duration_s":0.002718,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002072,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000646,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03497,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046624,"completed_at":"2026-09-13T10:52:39.713Z","load_balancing_strategy":"primary","db_duration_s":0.005463,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.243460] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.716Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f6b8cd2aa66c456b27e93ec0","created_at":"2026-09-13T10:52:39.200Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.203Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f6b8cd2aa66c456b27e93ec0: start","job_status":"start","queue_duration_s":0.512332,"scheduling_latency_s":0.512369} gitlab # [ 435.259006] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.762Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"f6b8cd2aa66c456b27e93ec0","created_at":"2026-09-13T10:52:39.200Z","trace_propagation_headers":{"sentry-trace":"95266026fa204c78a6be6c6fefa367ea-4ae09dd8ef8e4724","baggage":"sentry-trace_id=95266026fa204c78a6be6c6fefa367ea,sentry-release=Unknown"},"correlation_id":"01M2D6BJDB846B29EXYAVN44QE","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.203Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f6b8cd2aa66c456b27e93ec0: done: 0.0465 sec","job_status":"done","queue_duration_s":0.512332,"scheduling_latency_s":0.512369,"redis_calls":8,"redis_duration_s":0.002553,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00206,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0465,"completed_at":"2026-09-13T10:52:39.762Z","load_balancing_strategy":"primary","db_duration_s":0.004947,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.300104] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.765Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d78c7da08087597637171621","created_at":"2026-09-13T10:52:39.377Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.384Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d78c7da08087597637171621: start","job_status":"start","queue_duration_s":0.380488,"scheduling_latency_s":0.380523} gitlab # [ 435.316687] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.822Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"d78c7da08087597637171621","created_at":"2026-09-13T10:52:39.377Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.384Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-d78c7da08087597637171621: done: 0.057293 sec","job_status":"done","queue_duration_s":0.380488,"scheduling_latency_s":0.380523,"redis_calls":7,"redis_duration_s":0.002803,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002056,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000747,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.057293,"completed_at":"2026-09-13T10:52:39.822Z","load_balancing_strategy":"primary","db_duration_s":0.009985,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.359101] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.824Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c3a31785749cb2f108960ece","created_at":"2026-09-13T10:52:39.385Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.390Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c3a31785749cb2f108960ece: start","job_status":"start","queue_duration_s":0.434822,"scheduling_latency_s":0.434858} gitlab # [ 435.376461] gitaly[891]: time="2026-09-13T10:52:40.145Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:10.042" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:40.143" grpc.time_ms=1.708 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.388965] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.972Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"c3a31785749cb2f108960ece","created_at":"2026-09-13T10:52:39.385Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.390Z","job_size_bytes":65,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-c3a31785749cb2f108960ece: done: 0.147489 sec","job_status":"done","queue_duration_s":0.434822,"scheduling_latency_s":0.434858,"redis_calls":10,"redis_duration_s":0.004477999999999999,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002512,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001966,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.089454,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.147489,"completed_at":"2026-09-13T10:52:39.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.433947] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:39.975Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5ebec3e397f444095a780d2c","created_at":"2026-09-13T10:52:39.478Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.482Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5ebec3e397f444095a780d2c: start","job_status":"start","queue_duration_s":0.492995,"scheduling_latency_s":0.493051} gitlab # [ 435.449679] gitaly[891]: time="2026-09-13T10:52:40.216Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BM8YK5TBGM8J3PZ1WH8B grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:10.113" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:40.214" grpc.time_ms=1.377 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.461733] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:40.237Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"5ebec3e397f444095a780d2c","created_at":"2026-09-13T10:52:39.478Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-ca44ee9603e9406d","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:39.482Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-5ebec3e397f444095a780d2c: done: 0.2612 sec","job_status":"done","queue_duration_s":0.492995,"scheduling_latency_s":0.493051,"gitaly_calls":2,"gitaly_duration_s":0.010675,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.013881,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002244,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00197,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000499,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002999,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000507,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.001683,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001735,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.005,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.20215,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19104626,"duration_s":0.2612,"completed_at":"2026-09-13T10:52:40.236Z","load_balancing_strategy":"primary","db_duration_s":0.047232,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.512699] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:40.239Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ffd459580308185277d82a14","created_at":"2026-09-13T10:52:40.111Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-9de033e27b494343","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:40.115Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ffd459580308185277d82a14: start","job_status":"start","queue_duration_s":0.123799,"scheduling_latency_s":0.123839} gitlab # [ 435.528721] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:40.293Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"ffd459580308185277d82a14","created_at":"2026-09-13T10:52:40.111Z","trace_propagation_headers":{"sentry-trace":"a13b92d0161d43f7b8b7aeb6a017976c-9de033e27b494343","baggage":"sentry-trace_id=a13b92d0161d43f7b8b7aeb6a017976c,sentry-release=Unknown"},"correlation_id":"01M2D6BM8YK5TBGM8J3PZ1WH8B","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:40.115Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ffd459580308185277d82a14: done: 0.053342 sec","job_status":"done","queue_duration_s":0.123799,"scheduling_latency_s":0.123839,"redis_calls":7,"redis_duration_s":0.002253,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001643,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00061,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038079,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053342,"completed_at":"2026-09-13T10:52:40.293Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0087,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 435.815129] gitaly[891]: time="2026-09-13T10:52:40.689Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BNR6FMZ8FHTCPVTJ47HC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:10.186" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:40.687" grpc.time_ms=1.386 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.25 seconds) gitlab # [ 435.865296] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:40 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 156 gitlab # [ 436.386913] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:41.260Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c66a9e77d37b3168705287d8","created_at":"2026-09-13T10:52:41.230Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.256Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c66a9e77d37b3168705287d8: start","job_status":"start","queue_duration_s":0.003989,"scheduling_latency_s":0.004047} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 437.026571] gitaly[891]: time="2026-09-13T10:52:41.900Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BPY53WP3BXYBRP4CGMFH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:11.398" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:41.899" grpc.time_ms=1.356 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.22 seconds) gitlab # [ 437.089206] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 165 gitlab # [ 437.127670] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"c66a9e77d37b3168705287d8","created_at":"2026-09-13T10:52:41.230Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.256Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-c66a9e77d37b3168705287d8: done: 0.741297 sec","job_status":"done","queue_duration_s":0.003989,"scheduling_latency_s":0.004047,"redis_calls":70,"redis_duration_s":0.240193,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.052628,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002686,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.179022,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.005857,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.28675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.741297,"completed_at":"2026-09-13T10:52:42.001Z","load_balancing_strategy":"primary","db_duration_s":0.007625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.212751] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.011Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d2862b104bc23f2c7f034fc4","created_at":"2026-09-13T10:52:41.298Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d45fabf99d1bdea4c6993b91de30c6d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.309Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d2862b104bc23f2c7f034fc4: start","job_status":"start","queue_duration_s":0.702067,"scheduling_latency_s":0.702118} gitlab # [ 437.531192] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"d2862b104bc23f2c7f034fc4","created_at":"2026-09-13T10:52:41.298Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d45fabf99d1bdea4c6993b91de30c6d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.309Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-d2862b104bc23f2c7f034fc4: done: 0.392811 sec","job_status":"done","queue_duration_s":0.702067,"scheduling_latency_s":0.702118,"redis_calls":13,"redis_duration_s":0.028488000000000003,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003697,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008769,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.016022,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.026,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108195,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00487804,"exclusive_lock_hold_duration_s":0.29331116,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.392811,"completed_at":"2026-09-13T10:52:42.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.228455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.628370] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.441Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bd35676025e2759038eadb57","created_at":"2026-09-13T10:52:41.316Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9b8c03e4dbb74ae37609c109a3e2993","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.338Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bd35676025e2759038eadb57: start","job_status":"start","queue_duration_s":1.102815,"scheduling_latency_s":1.102865} gitlab # [ 437.695484] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"bd35676025e2759038eadb57","created_at":"2026-09-13T10:52:41.316Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9b8c03e4dbb74ae37609c109a3e2993","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.338Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-bd35676025e2759038eadb57: done: 0.128156 sec","job_status":"done","queue_duration_s":1.102815,"scheduling_latency_s":1.102865,"redis_calls":11,"redis_duration_s":0.025196,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005856,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012683,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00174774,"exclusive_lock_hold_duration_s":0.0077403,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.128156,"completed_at":"2026-09-13T10:52:42.569Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006954,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.786487] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.579Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f905bcd8088afc94b1efda3e","created_at":"2026-09-13T10:52:41.390Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b7c5d44c9b1a986f990ee1b56689fd1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.408Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f905bcd8088afc94b1efda3e: start","job_status":"start","queue_duration_s":1.170663,"scheduling_latency_s":1.170709} gitlab # [ 437.810098] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.685Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 437.875925] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.750Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f905bcd8088afc94b1efda3e","created_at":"2026-09-13T10:52:41.390Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2b7c5d44c9b1a986f990ee1b56689fd1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.408Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f905bcd8088afc94b1efda3e: done: 0.170908 sec","job_status":"done","queue_duration_s":1.170663,"scheduling_latency_s":1.170709,"redis_calls":9,"redis_duration_s":0.061036,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.031668,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.026882,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002486,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039655,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.170908,"completed_at":"2026-09-13T10:52:42.749Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.966703] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8e4ef51fecd3af0dc406e4cd","created_at":"2026-09-13T10:52:41.429Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e42b21f7734bbbcb6df6d5429b514f32","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.463Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8e4ef51fecd3af0dc406e4cd: start","job_status":"start","queue_duration_s":1.296695,"scheduling_latency_s":1.296741} gitlab # [ 437.984181] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.816Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 437.987649] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.836Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"8e4ef51fecd3af0dc406e4cd","created_at":"2026-09-13T10:52:41.429Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e42b21f7734bbbcb6df6d5429b514f32","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.463Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-8e4ef51fecd3af0dc406e4cd: done: 0.076141 sec","job_status":"done","queue_duration_s":1.296695,"scheduling_latency_s":1.296741,"redis_calls":9,"redis_duration_s":0.0042439999999999995,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000946,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029898,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076141,"completed_at":"2026-09-13T10:52:42.836Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.030482] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.839Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d5179db96babcf5fce8b6511","created_at":"2026-09-13T10:52:41.494Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb2164bec452ee4025513aa73f1557e2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.520Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d5179db96babcf5fce8b6511: start","job_status":"start","queue_duration_s":1.318495,"scheduling_latency_s":1.318531} gitlab # [ 438.044547] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.887Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"d5179db96babcf5fce8b6511","created_at":"2026-09-13T10:52:41.494Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cb2164bec452ee4025513aa73f1557e2","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.520Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-d5179db96babcf5fce8b6511: done: 0.047978 sec","job_status":"done","queue_duration_s":1.318495,"scheduling_latency_s":1.318531,"redis_calls":9,"redis_duration_s":0.002985,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000647,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001696,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047978,"completed_at":"2026-09-13T10:52:42.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007156,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 438.086223] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.889Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"24735d6bb9c6b63dcee08204","created_at":"2026-09-13T10:52:41.543Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"401c466d1a7a1ce6879c3539a808c7d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.568Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-24735d6bb9c6b63dcee08204: start","job_status":"start","queue_duration_s":1.321028,"scheduling_latency_s":1.321059} gitlab # [ 438.100963] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.906Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 438.104158] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.925Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"24735d6bb9c6b63dcee08204","created_at":"2026-09-13T10:52:41.543Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"401c466d1a7a1ce6879c3539a808c7d6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.568Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-24735d6bb9c6b63dcee08204: done: 0.036661 sec","job_status":"done","queue_duration_s":1.321028,"scheduling_latency_s":1.321059,"redis_calls":9,"redis_duration_s":0.003382,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000623,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001947,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000812,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019669,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036661,"completed_at":"2026-09-13T10:52:42.925Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000745,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.146752] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.928Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"f5d4c77628cddfbdae65b074","created_at":"2026-09-13T10:52:41.594Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e0cff23f6b0c7b80eeb42a37ab35eb91","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.619Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-f5d4c77628cddfbdae65b074: start","job_status":"start","queue_duration_s":1.309,"scheduling_latency_s":1.309042} gitlab # [ 438.160413] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.978Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"f5d4c77628cddfbdae65b074","created_at":"2026-09-13T10:52:41.594Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"e0cff23f6b0c7b80eeb42a37ab35eb91","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.619Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-f5d4c77628cddfbdae65b074: done: 0.049149 sec","job_status":"done","queue_duration_s":1.309,"scheduling_latency_s":1.309042,"redis_calls":10,"redis_duration_s":0.0038139999999999997,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000873,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002224,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000717,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025433,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063034,"duration_s":0.049149,"completed_at":"2026-09-13T10:52:42.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.226069] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:42.980Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"46dd3c6170cd94eff30ac7d1","created_at":"2026-09-13T10:52:41.642Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3693a2830ed31e1ae95ced7ba68d03b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.667Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-46dd3c6170cd94eff30ac7d1: start","job_status":"start","queue_duration_s":1.31306,"scheduling_latency_s":1.313097} gitlab # [ 438.248549] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.077Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"46dd3c6170cd94eff30ac7d1","created_at":"2026-09-13T10:52:41.642Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a3693a2830ed31e1ae95ced7ba68d03b","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.667Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-46dd3c6170cd94eff30ac7d1: done: 0.096485 sec","job_status":"done","queue_duration_s":1.31306,"scheduling_latency_s":1.313097,"redis_calls":11,"redis_duration_s":0.0044480000000000006,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.03,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00105536,"exclusive_lock_hold_duration_s":0.03639462,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.096485,"completed_at":"2026-09-13T10:52:43.077Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.298343] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.079Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6cda9b78f731d5e9365256bc","created_at":"2026-09-13T10:52:41.719Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86d61da99f99f4116d2b642af5e9d36f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.743Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6cda9b78f731d5e9365256bc: start","job_status":"start","queue_duration_s":1.336714,"scheduling_latency_s":1.336748} gitlab # [ 438.314066] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 111 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.23 seconds) gitlab # [ 438.316802] gitaly[891]: time="2026-09-13T10:52:43.116Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BR5GF9ZD4023C461HESQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:13.014" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:43.115" grpc.time_ms=1.071 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.663033] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"6cda9b78f731d5e9365256bc","created_at":"2026-09-13T10:52:41.719Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86d61da99f99f4116d2b642af5e9d36f","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.743Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-6cda9b78f731d5e9365256bc: done: 0.45697 sec","job_status":"done","queue_duration_s":1.336714,"scheduling_latency_s":1.336748,"redis_calls":564,"redis_duration_s":0.09470600000000001,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001167,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.093104,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.403761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.45697,"completed_at":"2026-09-13T10:52:43.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.759524] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c0a0edff7d588c63724f18bc","created_at":"2026-09-13T10:52:41.774Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4bbdc52699eeb1879912b8d595b93af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.803Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c0a0edff7d588c63724f18bc: start","job_status":"start","queue_duration_s":1.735054,"scheduling_latency_s":1.735073} gitlab # [ 438.779059] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.562Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c0a0edff7d588c63724f18bc","created_at":"2026-09-13T10:52:41.774Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c4bbdc52699eeb1879912b8d595b93af","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.803Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c0a0edff7d588c63724f18bc: done: 0.023715 sec","job_status":"done","queue_duration_s":1.735054,"scheduling_latency_s":1.735073,"redis_calls":11,"redis_duration_s":0.002729,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001009,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001232,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000488,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023715,"completed_at":"2026-09-13T10:52:43.562Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.823484] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.563Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ed4bbacd2c088cc84079a0e1","created_at":"2026-09-13T10:52:41.816Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569b30f83a8d738507e0eb2c2ae731de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.842Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ed4bbacd2c088cc84079a0e1: start","job_status":"start","queue_duration_s":1.72121,"scheduling_latency_s":1.721227} gitlab # [ 438.838351] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.583Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ed4bbacd2c088cc84079a0e1","created_at":"2026-09-13T10:52:41.816Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"569b30f83a8d738507e0eb2c2ae731de","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.842Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ed4bbacd2c088cc84079a0e1: done: 0.020101 sec","job_status":"done","queue_duration_s":1.72121,"scheduling_latency_s":1.721227,"redis_calls":9,"redis_duration_s":0.002351,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000556,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000531,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020101,"completed_at":"2026-09-13T10:52:43.583Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000623,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.880427] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.585Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2beff84c563b7175c4d79f70","created_at":"2026-09-13T10:52:41.869Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"911a18b64d6779d586b3dd3c1c911820","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.894Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2beff84c563b7175c4d79f70: start","job_status":"start","queue_duration_s":1.690472,"scheduling_latency_s":1.690488} gitlab # [ 438.894138] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.612Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2beff84c563b7175c4d79f70","created_at":"2026-09-13T10:52:41.869Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"911a18b64d6779d586b3dd3c1c911820","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.894Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2beff84c563b7175c4d79f70: done: 0.027098 sec","job_status":"done","queue_duration_s":1.690472,"scheduling_latency_s":1.690488,"redis_calls":9,"redis_duration_s":0.00236,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001311,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000517,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027098,"completed_at":"2026-09-13T10:52:43.612Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.936541] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.613Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bf6e60f264fadcc1b6daa0a","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bf6e60f264fadcc1b6daa0a: start","job_status":"start","queue_duration_s":1.679649,"scheduling_latency_s":1.679668} gitlab # [ 438.948993] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.635Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bf6e60f264fadcc1b6daa0a","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bf6e60f264fadcc1b6daa0a: done: 0.022264 sec","job_status":"done","queue_duration_s":1.679649,"scheduling_latency_s":1.679668,"redis_calls":15,"redis_duration_s":0.0034159999999999998,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000527,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022264,"completed_at":"2026-09-13T10:52:43.635Z","load_balancing_strategy":"primary","db_duration_s":0.000761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.989738] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b36d74649b992235d292b57c","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b36d74649b992235d292b57c: start","job_status":"start","queue_duration_s":1.703315,"scheduling_latency_s":1.703331} gitlab # [ 439.002007] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.653Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b36d74649b992235d292b57c","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b36d74649b992235d292b57c: done: 0.015936 sec","job_status":"done","queue_duration_s":1.703315,"scheduling_latency_s":1.703331,"redis_calls":13,"redis_duration_s":0.002757,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001819,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009225,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015936,"completed_at":"2026-09-13T10:52:43.653Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.041193] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911d3c3da64fd5e43afeb2b3","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911d3c3da64fd5e43afeb2b3: start","job_status":"start","queue_duration_s":1.720621,"scheduling_latency_s":1.720636} gitlab # [ 439.053349] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.669Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911d3c3da64fd5e43afeb2b3","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911d3c3da64fd5e43afeb2b3: done: 0.015302 sec","job_status":"done","queue_duration_s":1.720621,"scheduling_latency_s":1.720636,"redis_calls":13,"redis_duration_s":0.0027749999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000991,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001784,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009654,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015302,"completed_at":"2026-09-13T10:52:43.669Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.092322] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.671Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"945ad8de477bb3f762928e4b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-945ad8de477bb3f762928e4b: start","job_status":"start","queue_duration_s":1.737303,"scheduling_latency_s":1.737319} gitlab # [ 439.104668] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.687Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"945ad8de477bb3f762928e4b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-945ad8de477bb3f762928e4b: done: 0.015956 sec","job_status":"done","queue_duration_s":1.737303,"scheduling_latency_s":1.737319,"redis_calls":13,"redis_duration_s":0.00294,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000947,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015956,"completed_at":"2026-09-13T10:52:43.687Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.144368] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6baaaf30f2317f3e62964a9","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6baaaf30f2317f3e62964a9: start","job_status":"start","queue_duration_s":1.754778,"scheduling_latency_s":1.754795} gitlab # [ 439.156383] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6baaaf30f2317f3e62964a9","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6baaaf30f2317f3e62964a9: done: 0.015602 sec","job_status":"done","queue_duration_s":1.754778,"scheduling_latency_s":1.754795,"redis_calls":13,"redis_duration_s":0.0027879999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015602,"completed_at":"2026-09-13T10:52:43.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.195581] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6187cfcd037e3eefa3f724f","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6187cfcd037e3eefa3f724f: start","job_status":"start","queue_duration_s":1.771784,"scheduling_latency_s":1.771799} gitlab # [ 439.208263] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6187cfcd037e3eefa3f724f","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6187cfcd037e3eefa3f724f: done: 0.015547 sec","job_status":"done","queue_duration_s":1.771784,"scheduling_latency_s":1.771799,"redis_calls":13,"redis_duration_s":0.002797,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000932,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00981,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015547,"completed_at":"2026-09-13T10:52:43.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.247128] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"513850f0d5efab5bc1fe8707","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-513850f0d5efab5bc1fe8707: start","job_status":"start","queue_duration_s":1.788738,"scheduling_latency_s":1.788754} gitlab # [ 439.259557] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"513850f0d5efab5bc1fe8707","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-513850f0d5efab5bc1fe8707: done: 0.024578 sec","job_status":"done","queue_duration_s":1.788738,"scheduling_latency_s":1.788754,"redis_calls":13,"redis_duration_s":0.0064329999999999995,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001685,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004748,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024578,"completed_at":"2026-09-13T10:52:43.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.298545] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcab6ba01c0d820d46b7ee1","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcab6ba01c0d820d46b7ee1: start","job_status":"start","queue_duration_s":1.814814,"scheduling_latency_s":1.814831} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 439.311073] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6dcab6ba01c0d820d46b7ee1","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6dcab6ba01c0d820d46b7ee1: done: 0.015366 sec","job_status":"done","queue_duration_s":1.814814,"scheduling_latency_s":1.814831,"redis_calls":13,"redis_duration_s":0.002728,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000946,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015366,"completed_at":"2026-09-13T10:52:43.764Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.349137] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5de7cfd2608013c4fbf6714","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5de7cfd2608013c4fbf6714: start","job_status":"start","queue_duration_s":1.831514,"scheduling_latency_s":1.831531} gitlab # [ 439.361437] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.781Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5de7cfd2608013c4fbf6714","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5de7cfd2608013c4fbf6714: done: 0.016147 sec","job_status":"done","queue_duration_s":1.831514,"scheduling_latency_s":1.831531,"redis_calls":13,"redis_duration_s":0.002822,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000953,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016147,"completed_at":"2026-09-13T10:52:43.781Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.401016] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8373fae768821fbf093acd5","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8373fae768821fbf093acd5: start","job_status":"start","queue_duration_s":1.848972,"scheduling_latency_s":1.848988} gitlab # [ 439.413257] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8373fae768821fbf093acd5","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8373fae768821fbf093acd5: done: 0.015741 sec","job_status":"done","queue_duration_s":1.848972,"scheduling_latency_s":1.848988,"redis_calls":13,"redis_duration_s":0.003242,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001342,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015741,"completed_at":"2026-09-13T10:52:43.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.452368] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fbe4092342c33089e16577c","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fbe4092342c33089e16577c: start","job_status":"start","queue_duration_s":1.866023,"scheduling_latency_s":1.866041} gitlab # [ 439.464546] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fbe4092342c33089e16577c","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fbe4092342c33089e16577c: done: 0.016169 sec","job_status":"done","queue_duration_s":1.866023,"scheduling_latency_s":1.866041,"redis_calls":13,"redis_duration_s":0.002928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016169,"completed_at":"2026-09-13T10:52:43.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.503515] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26f005734a0c70c9f4ba19e7","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26f005734a0c70c9f4ba19e7: start","job_status":"start","queue_duration_s":1.883557,"scheduling_latency_s":1.883573} gitlab # [ 439.516686] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26f005734a0c70c9f4ba19e7","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26f005734a0c70c9f4ba19e7: done: 0.015062 sec","job_status":"done","queue_duration_s":1.883557,"scheduling_latency_s":1.883573,"redis_calls":13,"redis_duration_s":0.002695,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001781,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009519,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015062,"completed_at":"2026-09-13T10:52:43.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.24 seconds) gitlab # [ 439.556573] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abc91d312b154c67c62383ef","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abc91d312b154c67c62383ef: start","job_status":"start","queue_duration_s":1.899972,"scheduling_latency_s":1.899988} gitlab # [ 439.568731] gitaly[891]: time="2026-09-13T10:52:44.368Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BSBK40GFKMYCY7CEW50D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:14.166" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:44.366" grpc.time_ms=1.219 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 439.581543] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 141 gitlab # [ 439.584089] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abc91d312b154c67c62383ef","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abc91d312b154c67c62383ef: done: 0.015269 sec","job_status":"done","queue_duration_s":1.899972,"scheduling_latency_s":1.899988,"redis_calls":13,"redis_duration_s":0.003041,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001215,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001826,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015269,"completed_at":"2026-09-13T10:52:43.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.622335] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d63001b0fd0344d6b88166cf","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d63001b0fd0344d6b88166cf: start","job_status":"start","queue_duration_s":1.916655,"scheduling_latency_s":1.916671} gitlab # [ 439.634613] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d63001b0fd0344d6b88166cf","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d63001b0fd0344d6b88166cf: done: 0.015382 sec","job_status":"done","queue_duration_s":1.916655,"scheduling_latency_s":1.916671,"redis_calls":13,"redis_duration_s":0.002806,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001865,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009083,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015382,"completed_at":"2026-09-13T10:52:43.866Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.673239] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b19938ab00a5fda53a36b6de","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b19938ab00a5fda53a36b6de: start","job_status":"start","queue_duration_s":1.933342,"scheduling_latency_s":1.933357} gitlab # [ 439.685341] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b19938ab00a5fda53a36b6de","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b19938ab00a5fda53a36b6de: done: 0.01506 sec","job_status":"done","queue_duration_s":1.933342,"scheduling_latency_s":1.933357,"redis_calls":13,"redis_duration_s":0.002729,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000896,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001833,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009488,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01506,"completed_at":"2026-09-13T10:52:43.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.724289] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e45f61100f84123bd6759e8","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e45f61100f84123bd6759e8: start","job_status":"start","queue_duration_s":1.949801,"scheduling_latency_s":1.949817} gitlab # [ 439.736711] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0e45f61100f84123bd6759e8","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0e45f61100f84123bd6759e8: done: 0.01605 sec","job_status":"done","queue_duration_s":1.949801,"scheduling_latency_s":1.949817,"redis_calls":13,"redis_duration_s":0.003452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001392,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00206,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01605,"completed_at":"2026-09-13T10:52:43.899Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.776428] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.901Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a0b1f7efd1a4b78f0de3fd8","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a0b1f7efd1a4b78f0de3fd8: start","job_status":"start","queue_duration_s":1.967548,"scheduling_latency_s":1.967564} gitlab # [ 439.788755] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.916Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a0b1f7efd1a4b78f0de3fd8","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a0b1f7efd1a4b78f0de3fd8: done: 0.015006 sec","job_status":"done","queue_duration_s":1.967548,"scheduling_latency_s":1.967564,"redis_calls":13,"redis_duration_s":0.002692,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00091,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001782,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009426,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015006,"completed_at":"2026-09-13T10:52:43.916Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.827732] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.917Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2eabdae12fd1f6eef7390b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2eabdae12fd1f6eef7390b: start","job_status":"start","queue_duration_s":1.983886,"scheduling_latency_s":1.983902} gitlab # [ 439.840143] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2eabdae12fd1f6eef7390b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2eabdae12fd1f6eef7390b: done: 0.015105 sec","job_status":"done","queue_duration_s":1.983886,"scheduling_latency_s":1.983902,"redis_calls":13,"redis_duration_s":0.0025369999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00162,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015105,"completed_at":"2026-09-13T10:52:43.932Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.878970] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.934Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cf19c1cf7f3fc41d456711f","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cf19c1cf7f3fc41d456711f: start","job_status":"start","queue_duration_s":2.000426,"scheduling_latency_s":2.000441} gitlab # [ 439.891544] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.949Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cf19c1cf7f3fc41d456711f","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cf19c1cf7f3fc41d456711f: done: 0.015213 sec","job_status":"done","queue_duration_s":2.000426,"scheduling_latency_s":2.000441,"redis_calls":13,"redis_duration_s":0.00307,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015213,"completed_at":"2026-09-13T10:52:43.949Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.930525] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.951Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fa5b8e68f899100051d7ca","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fa5b8e68f899100051d7ca: start","job_status":"start","queue_duration_s":2.016953,"scheduling_latency_s":2.016969} gitlab # [ 439.942706] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.966Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6fa5b8e68f899100051d7ca","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6fa5b8e68f899100051d7ca: done: 0.015332 sec","job_status":"done","queue_duration_s":2.016953,"scheduling_latency_s":2.016969,"redis_calls":13,"redis_duration_s":0.002654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001703,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015332,"completed_at":"2026-09-13T10:52:43.966Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 439.982766] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.967Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14af42d71c17bb4fa3417a5a","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14af42d71c17bb4fa3417a5a: start","job_status":"start","queue_duration_s":2.033615,"scheduling_latency_s":2.033631} gitlab # [ 439.995015] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.982Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14af42d71c17bb4fa3417a5a","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14af42d71c17bb4fa3417a5a: done: 0.015161 sec","job_status":"done","queue_duration_s":2.033615,"scheduling_latency_s":2.033631,"redis_calls":13,"redis_duration_s":0.002654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001757,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015161,"completed_at":"2026-09-13T10:52:43.982Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.033730] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.984Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e8436f833dea063968a2630","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e8436f833dea063968a2630: start","job_status":"start","queue_duration_s":2.05004,"scheduling_latency_s":2.050056} gitlab # [ 440.046002] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:43.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4e8436f833dea063968a2630","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4e8436f833dea063968a2630: done: 0.015205 sec","job_status":"done","queue_duration_s":2.05004,"scheduling_latency_s":2.050056,"redis_calls":13,"redis_duration_s":0.0026360000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001724,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009509,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015205,"completed_at":"2026-09-13T10:52:43.999Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.085038] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.000Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82e67214cdd88b0b67c9fd44","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82e67214cdd88b0b67c9fd44: start","job_status":"start","queue_duration_s":2.066589,"scheduling_latency_s":2.066604} gitlab # [ 440.097449] nix-daemon[3760]: accepted connection from pid 4819, user nixbot gitlab # [ 440.098373] nix-daemon[3760]: accepted connection from pid 4781, user nixbot gitlab # [ 440.099300] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.015Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"82e67214cdd88b0b67c9fd44","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-82e67214cdd88b0b67c9fd44: done: 0.01528 sec","job_status":"done","queue_duration_s":2.066589,"scheduling_latency_s":2.066604,"redis_calls":13,"redis_duration_s":0.002836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000949,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001887,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00957,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01528,"completed_at":"2026-09-13T10:52:44.015Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.137145] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.017Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b199cb28838b3d50f350190","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b199cb28838b3d50f350190: start","job_status":"start","queue_duration_s":2.083189,"scheduling_latency_s":2.083205} gitlab # [ 440.149204] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.032Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4b199cb28838b3d50f350190","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4b199cb28838b3d50f350190: done: 0.015175 sec","job_status":"done","queue_duration_s":2.083189,"scheduling_latency_s":2.083205,"redis_calls":13,"redis_duration_s":0.002666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009035,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015175,"completed_at":"2026-09-13T10:52:44.032Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.187994] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.033Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6175853ff2c84f9b4533b5b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6175853ff2c84f9b4533b5b: start","job_status":"start","queue_duration_s":2.099775,"scheduling_latency_s":2.099791} gitlab # [ 440.199790] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f6175853ff2c84f9b4533b5b","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f6175853ff2c84f9b4533b5b: done: 0.015051 sec","job_status":"done","queue_duration_s":2.099775,"scheduling_latency_s":2.099791,"redis_calls":13,"redis_duration_s":0.0030429999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002126,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015051,"completed_at":"2026-09-13T10:52:44.048Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.237970] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.050Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cb8610331b1fedd54b74394","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cb8610331b1fedd54b74394: start","job_status":"start","queue_duration_s":2.116775,"scheduling_latency_s":2.116791} gitlab # [ 440.249996] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.067Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2cb8610331b1fedd54b74394","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2cb8610331b1fedd54b74394: done: 0.016537 sec","job_status":"done","queue_duration_s":2.116775,"scheduling_latency_s":2.116791,"redis_calls":13,"redis_duration_s":0.002776,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016537,"completed_at":"2026-09-13T10:52:44.067Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.288152] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97385969fcb70b6fa736f6fb","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97385969fcb70b6fa736f6fb: start","job_status":"start","queue_duration_s":2.134593,"scheduling_latency_s":2.134608} gitlab # [ 440.300088] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.085Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97385969fcb70b6fa736f6fb","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97385969fcb70b6fa736f6fb: done: 0.017346 sec","job_status":"done","queue_duration_s":2.134593,"scheduling_latency_s":2.134608,"redis_calls":13,"redis_duration_s":0.0030099999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017346,"completed_at":"2026-09-13T10:52:44.085Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.337668] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.087Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b5fed039eb3474bce7b542d","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b5fed039eb3474bce7b542d: start","job_status":"start","queue_duration_s":2.153191,"scheduling_latency_s":2.153207} gitlab # [ 440.349781] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b5fed039eb3474bce7b542d","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b5fed039eb3474bce7b542d: done: 0.01742 sec","job_status":"done","queue_duration_s":2.153191,"scheduling_latency_s":2.153207,"redis_calls":13,"redis_duration_s":0.0032310000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01742,"completed_at":"2026-09-13T10:52:44.104Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.389061] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.106Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75f913d6eeba80d1f8cc904","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75f913d6eeba80d1f8cc904: start","job_status":"start","queue_duration_s":2.171952,"scheduling_latency_s":2.171969} gitlab # [ 440.400910] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75f913d6eeba80d1f8cc904","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75f913d6eeba80d1f8cc904: done: 0.017737 sec","job_status":"done","queue_duration_s":2.171952,"scheduling_latency_s":2.171969,"redis_calls":13,"redis_duration_s":0.004106,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002568,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017737,"completed_at":"2026-09-13T10:52:44.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.438694] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b90c5f4dede52376fdaf8b17","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b90c5f4dede52376fdaf8b17: start","job_status":"start","queue_duration_s":2.191043,"scheduling_latency_s":2.191058} gitlab # [ 440.450829] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b90c5f4dede52376fdaf8b17","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b90c5f4dede52376fdaf8b17: done: 0.027387 sec","job_status":"done","queue_duration_s":2.191043,"scheduling_latency_s":2.191058,"redis_calls":13,"redis_duration_s":0.00378,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00227,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027387,"completed_at":"2026-09-13T10:52:44.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.488450] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6919a1756c516854faa6ff0e","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6919a1756c516854faa6ff0e: start","job_status":"start","queue_duration_s":2.22132,"scheduling_latency_s":2.22136} gitlab # [ 440.500182] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.195Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6919a1756c516854faa6ff0e","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6919a1756c516854faa6ff0e: done: 0.040569 sec","job_status":"done","queue_duration_s":2.22132,"scheduling_latency_s":2.22136,"redis_calls":13,"redis_duration_s":0.004439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001464,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.040569,"completed_at":"2026-09-13T10:52:44.195Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.537600] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6711e722917578278006bfc","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6711e722917578278006bfc: start","job_status":"start","queue_duration_s":2.265914,"scheduling_latency_s":2.265954} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 440.549614] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6711e722917578278006bfc","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6711e722917578278006bfc: done: 0.032532 sec","job_status":"done","queue_duration_s":2.265914,"scheduling_latency_s":2.265954,"redis_calls":13,"redis_duration_s":0.0036049999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001204,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002401,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021193,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.032532,"completed_at":"2026-09-13T10:52:44.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.586036] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5693e42c17ae5c918e0595","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5693e42c17ae5c918e0595: start","job_status":"start","queue_duration_s":2.301265,"scheduling_latency_s":2.30129} gitlab # [ 440.597972] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.259Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a5693e42c17ae5c918e0595","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a5693e42c17ae5c918e0595: done: 0.024617 sec","job_status":"done","queue_duration_s":2.301265,"scheduling_latency_s":2.30129,"redis_calls":13,"redis_duration_s":0.002206,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000847,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001359,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024617,"completed_at":"2026-09-13T10:52:44.259Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.636302] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e04493a46a9b8e57675253c2","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e04493a46a9b8e57675253c2: start","job_status":"start","queue_duration_s":2.328511,"scheduling_latency_s":2.328538} gitlab # [ 440.648628] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e04493a46a9b8e57675253c2","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e04493a46a9b8e57675253c2: done: 0.022446 sec","job_status":"done","queue_duration_s":2.328511,"scheduling_latency_s":2.328538,"redis_calls":13,"redis_duration_s":0.0019529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001258,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022446,"completed_at":"2026-09-13T10:52:44.284Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.687693] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb484c69391bc568808195e","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb484c69391bc568808195e: start","job_status":"start","queue_duration_s":2.353217,"scheduling_latency_s":2.353237} gitlab # [ 440.699962] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.308Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3bb484c69391bc568808195e","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3bb484c69391bc568808195e: done: 0.02083 sec","job_status":"done","queue_duration_s":2.353217,"scheduling_latency_s":2.353237,"redis_calls":13,"redis_duration_s":0.0019069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001205,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02083,"completed_at":"2026-09-13T10:52:44.308Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.739552] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0b7704653c4f275d86ee697","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0b7704653c4f275d86ee697: start","job_status":"start","queue_duration_s":2.376169,"scheduling_latency_s":2.376187} gitlab # [ 440.752046] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0b7704653c4f275d86ee697","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0b7704653c4f275d86ee697: done: 0.017668 sec","job_status":"done","queue_duration_s":2.376169,"scheduling_latency_s":2.376187,"redis_calls":13,"redis_duration_s":0.001548,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017668,"completed_at":"2026-09-13T10:52:44.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.791510] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc7964235b02ae7097ad446","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc7964235b02ae7097ad446: start","job_status":"start","queue_duration_s":2.395636,"scheduling_latency_s":2.395655} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.27 seconds) gitlab # [ 440.805227] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.346Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2bc7964235b02ae7097ad446","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2bc7964235b02ae7097ad446: done: 0.016325 sec","job_status":"done","queue_duration_s":2.395636,"scheduling_latency_s":2.395655,"redis_calls":13,"redis_duration_s":0.001699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000615,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001084,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016325,"completed_at":"2026-09-13T10:52:44.345Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.844082] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.347Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c735a6e039f7a4c60bdac686","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c735a6e039f7a4c60bdac686: start","job_status":"start","queue_duration_s":2.413046,"scheduling_latency_s":2.413063} gitlab # [ 440.856580] gitaly[891]: time="2026-09-13T10:52:45.647Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BTK7E5B1PW34SWCB5XS7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:15.444" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:45.645" grpc.time_ms=1.282 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 440.869316] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:45 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 156 gitlab # [ 440.871906] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.363Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c735a6e039f7a4c60bdac686","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c735a6e039f7a4c60bdac686: done: 0.016701 sec","job_status":"done","queue_duration_s":2.413046,"scheduling_latency_s":2.413063,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016701,"completed_at":"2026-09-13T10:52:44.363Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.910412] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.364Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"686419c223eb4831dac4a051","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-686419c223eb4831dac4a051: start","job_status":"start","queue_duration_s":2.430789,"scheduling_latency_s":2.430804} gitlab # [ 440.922790] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.381Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"686419c223eb4831dac4a051","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-686419c223eb4831dac4a051: done: 0.016915 sec","job_status":"done","queue_duration_s":2.430789,"scheduling_latency_s":2.430804,"redis_calls":13,"redis_duration_s":0.0016089999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016915,"completed_at":"2026-09-13T10:52:44.381Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 440.961135] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.382Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd35f00b644bd26b1c248cb3","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd35f00b644bd26b1c248cb3: start","job_status":"start","queue_duration_s":2.448819,"scheduling_latency_s":2.448837} gitlab # [ 440.973432] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd35f00b644bd26b1c248cb3","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd35f00b644bd26b1c248cb3: done: 0.016042 sec","job_status":"done","queue_duration_s":2.448819,"scheduling_latency_s":2.448837,"redis_calls":13,"redis_duration_s":0.001803,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000763,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00104,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016042,"completed_at":"2026-09-13T10:52:44.398Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.012596] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.400Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e99335099e1ff4b46ccb3def","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e99335099e1ff4b46ccb3def: start","job_status":"start","queue_duration_s":2.465925,"scheduling_latency_s":2.465941} gitlab # [ 441.024564] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e99335099e1ff4b46ccb3def","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e99335099e1ff4b46ccb3def: done: 0.015496 sec","job_status":"done","queue_duration_s":2.465925,"scheduling_latency_s":2.465941,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015496,"completed_at":"2026-09-13T10:52:44.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.065375] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.416Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc64e3c9013f28ee341d803","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc64e3c9013f28ee341d803: start","job_status":"start","queue_duration_s":2.482499,"scheduling_latency_s":2.482515} gitlab # [ 441.077627] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.432Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3fc64e3c9013f28ee341d803","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3fc64e3c9013f28ee341d803: done: 0.015746 sec","job_status":"done","queue_duration_s":2.482499,"scheduling_latency_s":2.482515,"redis_calls":13,"redis_duration_s":0.001563,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015746,"completed_at":"2026-09-13T10:52:44.432Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.116276] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27382901dc1e6e682005fea4","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27382901dc1e6e682005fea4: start","job_status":"start","queue_duration_s":2.499372,"scheduling_latency_s":2.499389} gitlab # [ 441.128232] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27382901dc1e6e682005fea4","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27382901dc1e6e682005fea4: done: 0.016248 sec","job_status":"done","queue_duration_s":2.499372,"scheduling_latency_s":2.499389,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011416,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016248,"completed_at":"2026-09-13T10:52:44.449Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.166743] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85f885f282412f326fb8fedb","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85f885f282412f326fb8fedb: start","job_status":"start","queue_duration_s":2.517073,"scheduling_latency_s":2.517092} gitlab # [ 441.178871] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85f885f282412f326fb8fedb","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85f885f282412f326fb8fedb: done: 0.016312 sec","job_status":"done","queue_duration_s":2.517073,"scheduling_latency_s":2.517092,"redis_calls":13,"redis_duration_s":0.0024980000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000693,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001805,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016312,"completed_at":"2026-09-13T10:52:44.467Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.218438] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"904d4f08fe021fcd273045c6","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-904d4f08fe021fcd273045c6: start","job_status":"start","queue_duration_s":2.53545,"scheduling_latency_s":2.535467} gitlab # [ 441.230701] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"904d4f08fe021fcd273045c6","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-904d4f08fe021fcd273045c6: done: 0.016041 sec","job_status":"done","queue_duration_s":2.53545,"scheduling_latency_s":2.535467,"redis_calls":13,"redis_duration_s":0.003057,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00184,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016041,"completed_at":"2026-09-13T10:52:44.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.269137] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e10c17dc7486c370c4b0e600","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e10c17dc7486c370c4b0e600: start","job_status":"start","queue_duration_s":2.553319,"scheduling_latency_s":2.553336} gitlab # [ 441.281388] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e10c17dc7486c370c4b0e600","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e10c17dc7486c370c4b0e600: done: 0.017096 sec","job_status":"done","queue_duration_s":2.553319,"scheduling_latency_s":2.553336,"redis_calls":13,"redis_duration_s":0.003707,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017096,"completed_at":"2026-09-13T10:52:44.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.319945] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36a0dacfab7b79530d7e8e87","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36a0dacfab7b79530d7e8e87: start","job_status":"start","queue_duration_s":2.572519,"scheduling_latency_s":2.572537} gitlab # [ 441.332221] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"36a0dacfab7b79530d7e8e87","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-36a0dacfab7b79530d7e8e87: done: 0.016895 sec","job_status":"done","queue_duration_s":2.572519,"scheduling_latency_s":2.572537,"redis_calls":13,"redis_duration_s":0.0028320000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011204,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016895,"completed_at":"2026-09-13T10:52:44.523Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.370507] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.524Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ca083c472d7f718abd8453","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ca083c472d7f718abd8453: start","job_status":"start","queue_duration_s":2.590732,"scheduling_latency_s":2.590748} gitlab # [ 441.382747] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.541Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ca083c472d7f718abd8453","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ca083c472d7f718abd8453: done: 0.01688 sec","job_status":"done","queue_duration_s":2.590732,"scheduling_latency_s":2.590748,"redis_calls":13,"redis_duration_s":0.00263,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00089,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00174,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01688,"completed_at":"2026-09-13T10:52:44.541Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.422141] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.543Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"198a8767f5895edf674dcd10","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-198a8767f5895edf674dcd10: start","job_status":"start","queue_duration_s":2.60898,"scheduling_latency_s":2.608996} gitlab # [ 441.434595] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.558Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"198a8767f5895edf674dcd10","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-198a8767f5895edf674dcd10: done: 0.015432 sec","job_status":"done","queue_duration_s":2.60898,"scheduling_latency_s":2.608996,"redis_calls":13,"redis_duration_s":0.002657,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001743,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015432,"completed_at":"2026-09-13T10:52:44.558Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.473046] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.559Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2fe8f199d660909a007aaf6","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2fe8f199d660909a007aaf6: start","job_status":"start","queue_duration_s":2.625757,"scheduling_latency_s":2.625773} gitlab # [ 441.485585] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.574Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d2fe8f199d660909a007aaf6","created_at":"2026-09-13T10:52:41.427Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-43394442aaf941de","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"6235c18ba2e866d188a3488a3efc206d","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:41.934Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d2fe8f199d660909a007aaf6: done: 0.014672 sec","job_status":"done","queue_duration_s":2.625757,"scheduling_latency_s":2.625773,"redis_calls":13,"redis_duration_s":0.002647,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009168,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014672,"completed_at":"2026-09-13T10:52:44.574Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.524482] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9ebf6b336d6d291457eb9792","created_at":"2026-09-13T10:52:41.924Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"960e0badd3e76df317003d3522d052d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:52:41.951Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9ebf6b336d6d291457eb9792: start","job_status":"start","queue_duration_s":2.624186,"scheduling_latency_s":2.624202} gitlab # [ 441.537712] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.587Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 441.541041] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"9ebf6b336d6d291457eb9792","created_at":"2026-09-13T10:52:41.924Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"960e0badd3e76df317003d3522d052d4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-13T10:52:41.951Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-9ebf6b336d6d291457eb9792: done: 0.019033 sec","job_status":"done","queue_duration_s":2.624186,"scheduling_latency_s":2.624202,"redis_calls":9,"redis_duration_s":0.002315,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000616,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00122,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019033,"completed_at":"2026-09-13T10:52:44.594Z","load_balancing_strategy":"primary","db_duration_s":0.000836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.582188] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.596Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"019ff0742c10a723f03792ab","created_at":"2026-09-13T10:52:41.996Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa023729f0945bdb084c056fcf2e5400","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.064Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-019ff0742c10a723f03792ab: start","job_status":"start","queue_duration_s":2.532008,"scheduling_latency_s":2.532025} gitlab # [ 441.596745] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.621Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"019ff0742c10a723f03792ab","created_at":"2026-09-13T10:52:41.996Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"aa023729f0945bdb084c056fcf2e5400","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.064Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-019ff0742c10a723f03792ab: done: 0.025378 sec","job_status":"done","queue_duration_s":2.532008,"scheduling_latency_s":2.532025,"redis_calls":11,"redis_duration_s":0.002857,"redis_read_bytes":9,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001201,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055792,"exclusive_lock_hold_duration_s":0.00629272,"duration_s":0.025378,"completed_at":"2026-09-13T10:52:44.621Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004191,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.640186] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.623Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"189f47e5109ac2eb743f1c85","created_at":"2026-09-13T10:52:42.294Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69dc8d1fbceaee1e7b935a62b2e310f5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.306Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-189f47e5109ac2eb743f1c85: start","job_status":"start","queue_duration_s":2.317052,"scheduling_latency_s":2.317068} gitlab # [ 441.654543] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.650Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"189f47e5109ac2eb743f1c85","created_at":"2026-09-13T10:52:42.294Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"69dc8d1fbceaee1e7b935a62b2e310f5","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.306Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-189f47e5109ac2eb743f1c85: done: 0.027808 sec","job_status":"done","queue_duration_s":2.317052,"scheduling_latency_s":2.317068,"redis_calls":9,"redis_duration_s":0.002361,"redis_read_bytes":7,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001341,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":536,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00047,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027808,"completed_at":"2026-09-13T10:52:44.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005228,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.697119] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.652Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"061feffe620cd4e4fdec87ab","created_at":"2026-09-13T10:52:42.325Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.343Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-061feffe620cd4e4fdec87ab: start","job_status":"start","queue_duration_s":2.308433,"scheduling_latency_s":2.308449} gitlab # [ 441.710413] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"061feffe620cd4e4fdec87ab","created_at":"2026-09-13T10:52:42.325Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.343Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-061feffe620cd4e4fdec87ab: done: 0.035042 sec","job_status":"done","queue_duration_s":2.308433,"scheduling_latency_s":2.308449,"redis_calls":18,"redis_duration_s":0.004491,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001108,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000317,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002316,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035042,"completed_at":"2026-09-13T10:52:44.687Z","load_balancing_strategy":"primary","db_duration_s":0.003075,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.752490] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.688Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f92b26c762d94f4a1efd5031","created_at":"2026-09-13T10:52:42.371Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d6d3374acce3d0e94d5a0c56a8a855a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.397Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f92b26c762d94f4a1efd5031: start","job_status":"start","queue_duration_s":2.291532,"scheduling_latency_s":2.291549} gitlab # [ 441.766386] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:44.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f92b26c762d94f4a1efd5031","created_at":"2026-09-13T10:52:42.371Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4d6d3374acce3d0e94d5a0c56a8a855a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.397Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f92b26c762d94f4a1efd5031: done: 0.311122 sec","job_status":"done","queue_duration_s":2.291532,"scheduling_latency_s":2.291549,"redis_calls":13,"redis_duration_s":0.0034809999999999997,"redis_read_bytes":11,"redis_write_bytes":1567,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000983,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.06,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.230708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058202,"exclusive_lock_hold_duration_s":0.28760894,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.311122,"completed_at":"2026-09-13T10:52:44.999Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.091116,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.810664] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.001Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9c7389a003280db7ab162c86","created_at":"2026-09-13T10:52:42.435Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee045338b113e225b77c4a20da59e9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:52:42.452Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9c7389a003280db7ab162c86: start","job_status":"start","queue_duration_s":2.549136,"scheduling_latency_s":2.549156} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 441.823637] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"9c7389a003280db7ab162c86","created_at":"2026-09-13T10:52:42.435Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4ee045338b113e225b77c4a20da59e9d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-13T10:52:42.452Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-9c7389a003280db7ab162c86: done: 0.032482 sec","job_status":"done","queue_duration_s":2.549136,"scheduling_latency_s":2.549156,"redis_calls":9,"redis_duration_s":0.002279,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000477,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001301,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000501,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019573,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032482,"completed_at":"2026-09-13T10:52:45.033Z","load_balancing_strategy":"primary","db_duration_s":0.005809,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.862376] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.035Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d4665a2977ad449c18faefed","created_at":"2026-09-13T10:52:42.462Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86fc81c9d90fd1f97dc414d60f281bab","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.473Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d4665a2977ad449c18faefed: start","job_status":"start","queue_duration_s":2.562163,"scheduling_latency_s":2.562182} gitlab # [ 441.874458] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.054Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"d4665a2977ad449c18faefed","created_at":"2026-09-13T10:52:42.462Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"86fc81c9d90fd1f97dc414d60f281bab","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.473Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-d4665a2977ad449c18faefed: done: 0.019079 sec","job_status":"done","queue_duration_s":2.562163,"scheduling_latency_s":2.562182,"redis_calls":11,"redis_duration_s":0.002735,"redis_read_bytes":8,"redis_write_bytes":1094,"redis_cache_calls":2,"redis_cache_duration_s":0.000499,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00055,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001221,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":482,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019079,"completed_at":"2026-09-13T10:52:45.054Z","load_balancing_strategy":"primary","db_duration_s":0.000691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.915656] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.055Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fcd550076e7729b72b4982ca","created_at":"2026-09-13T10:52:42.500Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f464fa296f976196039a3d24c14e019","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:52:42.529Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fcd550076e7729b72b4982ca: start","job_status":"start","queue_duration_s":2.526539,"scheduling_latency_s":2.526556} gitlab # [ 441.928965] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.067Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 441.932610] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.073Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"fcd550076e7729b72b4982ca","created_at":"2026-09-13T10:52:42.500Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f464fa296f976196039a3d24c14e019","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-13T10:52:42.529Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-fcd550076e7729b72b4982ca: done: 0.017957 sec","job_status":"done","queue_duration_s":2.526539,"scheduling_latency_s":2.526556,"redis_calls":9,"redis_duration_s":0.002098,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000488,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001171,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017957,"completed_at":"2026-09-13T10:52:45.073Z","load_balancing_strategy":"primary","db_duration_s":0.000573,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 441.973089] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.075Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3657060ff837506521a962f2","created_at":"2026-09-13T10:52:42.556Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff5272ea6c737e848416078a73628baf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.573Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3657060ff837506521a962f2: start","job_status":"start","queue_duration_s":2.501739,"scheduling_latency_s":2.501755} gitlab # [ 441.987564] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.086Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 441.990996] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.092Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3657060ff837506521a962f2","created_at":"2026-09-13T10:52:42.556Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ff5272ea6c737e848416078a73628baf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.573Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3657060ff837506521a962f2: done: 0.017581 sec","job_status":"done","queue_duration_s":2.501739,"scheduling_latency_s":2.501755,"redis_calls":9,"redis_duration_s":0.002086,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017581,"completed_at":"2026-09-13T10:52:45.092Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000535,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.033862] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.094Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"93a498c297d846ba9b7304f9","created_at":"2026-09-13T10:52:42.596Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9450e1872562892387fd261ed17531e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.615Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-93a498c297d846ba9b7304f9: start","job_status":"start","queue_duration_s":2.478901,"scheduling_latency_s":2.478917} gitlab # [ 442.048309] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.124Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"93a498c297d846ba9b7304f9","created_at":"2026-09-13T10:52:42.596Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a9450e1872562892387fd261ed17531e","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.615Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-93a498c297d846ba9b7304f9: done: 0.030354 sec","job_status":"done","queue_duration_s":2.478901,"scheduling_latency_s":2.478917,"redis_calls":12,"redis_duration_s":0.003421,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000471,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001532,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00894838,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.030354,"completed_at":"2026-09-13T10:52:45.124Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003805,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.27 seconds) gitlab # [ 442.091979] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.126Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"45ad84492022e1124b9073ca","created_at":"2026-09-13T10:52:42.669Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1271b036d3da253654cf95b5998ab323","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.688Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-45ad84492022e1124b9073ca: start","job_status":"start","queue_duration_s":2.438021,"scheduling_latency_s":2.438038} gitlab # [ 442.106342] gitaly[891]: time="2026-09-13T10:52:46.919Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BVV32SAMNTZG9BNXQAQ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:16.716" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:46.917" grpc.time_ms=1.288 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 442.118884] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:46 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 151 gitlab # [ 442.121600] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.165Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"45ad84492022e1124b9073ca","created_at":"2026-09-13T10:52:42.669Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1271b036d3da253654cf95b5998ab323","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.688Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-45ad84492022e1124b9073ca: done: 0.038986 sec","job_status":"done","queue_duration_s":2.438021,"scheduling_latency_s":2.438038,"redis_calls":9,"redis_duration_s":0.002332,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000482,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001328,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.038986,"completed_at":"2026-09-13T10:52:45.165Z","load_balancing_strategy":"primary","db_duration_s":0.018021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.163181] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.166Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7c2f9b0bf3673c3497f153e7","created_at":"2026-09-13T10:52:42.737Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"383e379ab7406cd1b9720b7bd37d0fab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.757Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7c2f9b0bf3673c3497f153e7: start","job_status":"start","queue_duration_s":2.409071,"scheduling_latency_s":2.40909} gitlab # [ 442.176443] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"7c2f9b0bf3673c3497f153e7","created_at":"2026-09-13T10:52:42.737Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-2947c46535c14f70","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"383e379ab7406cd1b9720b7bd37d0fab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:42.757Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-7c2f9b0bf3673c3497f153e7: done: 0.026891 sec","job_status":"done","queue_duration_s":2.409071,"scheduling_latency_s":2.40909,"redis_calls":11,"redis_duration_s":0.002818,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001253,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005368,"exclusive_lock_hold_duration_s":0.00792676,"duration_s":0.026891,"completed_at":"2026-09-13T10:52:45.193Z","load_balancing_strategy":"primary","db_duration_s":0.004914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.217013] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.195Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5efb01cd97c9d5ab9af03e34","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5efb01cd97c9d5ab9af03e34: start","job_status":"start","queue_duration_s":0.514452,"scheduling_latency_s":0.514471} gitlab # [ 442.229111] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.211Z","message":"No existing merge request to be cleaned up."} gitlab # [ 442.230728] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.225Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5efb01cd97c9d5ab9af03e34","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5efb01cd97c9d5ab9af03e34: done: 0.030063 sec","job_status":"done","queue_duration_s":0.514452,"scheduling_latency_s":0.514471,"redis_calls":12,"redis_duration_s":0.003402,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001884,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017787,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030063,"completed_at":"2026-09-13T10:52:45.225Z","load_balancing_strategy":"primary","db_duration_s":0.005111,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.270940] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.226Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ebc28b9f00b79649f531b571","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ebc28b9f00b79649f531b571: start","job_status":"start","queue_duration_s":0.545889,"scheduling_latency_s":0.545906} gitlab # [ 442.283624] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.237Z","message":"No existing merge request to be cleaned up."} gitlab # [ 442.285391] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.246Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ebc28b9f00b79649f531b571","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ebc28b9f00b79649f531b571: done: 0.01972 sec","job_status":"done","queue_duration_s":0.545889,"scheduling_latency_s":0.545906,"redis_calls":10,"redis_duration_s":0.002613,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001546,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01972,"completed_at":"2026-09-13T10:52:45.246Z","load_balancing_strategy":"primary","db_duration_s":0.002359,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.323761] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.247Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2102433f241c9f98e387aa22","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2102433f241c9f98e387aa22: start","job_status":"start","queue_duration_s":0.567136,"scheduling_latency_s":0.567153} gitlab # [ 442.336144] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.258Z","message":"No existing merge request to be cleaned up."} gitlab # [ 442.337782] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.266Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2102433f241c9f98e387aa22","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2102433f241c9f98e387aa22: done: 0.019223 sec","job_status":"done","queue_duration_s":0.567136,"scheduling_latency_s":0.567153,"redis_calls":10,"redis_duration_s":0.002407,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001009,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001398,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019223,"completed_at":"2026-09-13T10:52:45.266Z","load_balancing_strategy":"primary","db_duration_s":0.002324,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 442.375553] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.268Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"386a9eb21f221e09925c8105","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-386a9eb21f221e09925c8105: start","job_status":"start","queue_duration_s":0.587719,"scheduling_latency_s":0.587735} gitlab # [ 442.387982] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.279Z","message":"No existing merge request to be cleaned up."} gitlab # [ 442.389711] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:45.287Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"386a9eb21f221e09925c8105","created_at":"2026-09-13T10:52:44.668Z","trace_propagation_headers":{"sentry-trace":"14d6c8de74654b78a2bfceb5c5abc1fe-105258e04ad14d86","baggage":"sentry-trace_id=14d6c8de74654b78a2bfceb5c5abc1fe,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a9c29ae16fc35960a6273fd0ddfad479","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-13T10:52:44.680Z","job_size_bytes":2,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-386a9eb21f221e09925c8105: done: 0.019613 sec","job_status":"done","queue_duration_s":0.587719,"scheduling_latency_s":0.587735,"redis_calls":10,"redis_duration_s":0.002535,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001024,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001511,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019613,"completed_at":"2026-09-13T10:52:45.287Z","load_balancing_strategy":"primary","db_duration_s":0.002545,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 443.283865] gitaly[891]: time="2026-09-13T10:52:48.157Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BX1KTDM5T7PAS1825D9A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:18.055" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:48.156" grpc.time_ms=1.281 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.23 seconds) gitlab # [ 443.331528] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 153 gitlab # [ 444.177075] nix-daemon[3760]: accepted connection from pid 4781, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 444.368332] gitaly[891]: time="2026-09-13T10:52:49.242Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:19.139" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.240" grpc.time_ms=1.349 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.477046] gitaly[891]: time="2026-09-13T10:52:49.352Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:59.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.335" grpc.time_ms=16.114 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.509882] gitaly[891]: time="2026-09-13T10:52:49.369Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY7EXYCSEB79Z3HSGT97 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:19.167" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.368" grpc.time_ms=1.211 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.22 seconds) gitlab # [ 444.551122] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 162 gitlab # [ 444.571505] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:49.445Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"91815acd4220217b76476efa","created_at":"2026-09-13T10:52:49.433Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-672d787302a74fa2","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.441Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-91815acd4220217b76476efa: start","job_status":"start","queue_duration_s":0.00381,"scheduling_latency_s":0.003875} gitlab # [ 444.609533] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 355 gitlab # [ 444.615180] nixbot[3588]: {"ts": "2026-09-13T10:52:49.476Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 \"HTTP/1.1 201 Created\""} gitlab # [ 444.703851] gitaly[891]: time="2026-09-13T10:52:49.579Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BYEC5GTQRP11EQAP550A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:19.476" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.577" grpc.time_ms=1.184 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.804839] gitaly[891]: time="2026-09-13T10:52:49.679Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BYEC5GTQRP11EQAP550A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:59.269" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.660" grpc.time_ms=19.441 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.841175] gitaly[891]: time="2026-09-13T10:52:49.682Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:49.681" grpc.time_ms=0.768 pid=891 span.kind=server system=grpc gitlab # [ 444.859452] gitaly[891]: time="2026-09-13T10:52:49.728Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:49.728" grpc.time_ms=0.623 pid=891 span.kind=server system=grpc gitlab # [ 444.894490] gitaly[891]: time="2026-09-13T10:52:49.769Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-13T10:52:49.766" grpc.time_ms=3.129 pid=891 span.kind=server system=grpc gitlab # [ 444.919070] gitaly[891]: time="2026-09-13T10:52:49.776Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" operations="[repack_refs]" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.939396] gitaly[891]: time="2026-09-13T10:52:49.780Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 diskcache=cdcfca7a-3fd7-4c8b-b766-c9de93badc8f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.956921] nixbot[3588]: {"ts": "2026-09-13T10:52:49.773Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 \"HTTP/1.1 201 Created\""} gitlab # [ 444.961475] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:49 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 286 gitlab # [ 444.965417] gitaly[891]: time="2026-09-13T10:52:49.780Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=170 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7212 command.oublock=0 command.real_time_ms=158 command.spawn_token_fork_ms=1 command.system_time_ms=96 command.user_time_ms=74 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" grpc.time_ms=164.892 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 444.983460] gitaly[891]: time="2026-09-13T10:52:49.780Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" pid=891 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-13 10:50:57.19197254 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 444.998016] gitaly[891]: time="2026-09-13T10:52:49.814Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" pid=891 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 445.013043] gitaly[891]: time="2026-09-13T10:52:49.815Z" level=info msg="optimized repository" clean_stale_data_time=33.79856ms clean_work_trees_time="262.4µs" commit_graph_time=860ns correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.9µs" pid=891 prune_time=300ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="2.34µs" storage_name=default system=housekeeping total_time=34.11208ms gitlab # [ 445.019537] gitaly[891]: time="2026-09-13T10:52:49.815Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-13T10:53:19.514" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.615" pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.030115] gitaly[891]: time="2026-09-13T10:52:49.878Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BYR27AQMC45J1J5AX1R4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:19.676" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.877" grpc.time_ms=0.985 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.043888] gitaly[891]: time="2026-09-13T10:52:49.886Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-13T10:53:19.777" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.877" grpc.time_ms=8.941 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.067073] gitaly[891]: time="2026-09-13T10:52:49.942Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BY2XNE6GDWJBX1J393X0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:19.840" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:49.941" grpc.time_ms=0.781 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.084949] gitaly[891]: time="2026-09-13T10:52:49.944Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BYR27AQMC45J1J5AX1R4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-13T10:52:59.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-13T10:52:49.931" grpc.time_ms=12.599 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.103444] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:49.961Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"4279f081231c12338a0b538a","created_at":"2026-09-13T10:52:49.955Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"91815acd4220217b76476efa","job_size_bytes":3,"pid":3454,"job_status":"deduplicated","message":"PipelineProcessWorker JID-4279f081231c12338a0b538a: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60} gitlab # [ 445.121477] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:49.968Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"91815acd4220217b76476efa","created_at":"2026-09-13T10:52:49.433Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-672d787302a74fa2","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.441Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3454,"job_status":"rescheduled","message":"PipelineProcessWorker JID-91815acd4220217b76476efa: rescheduled"} gitlab # [ 445.138158] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:49.983Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"91815acd4220217b76476efa","created_at":"2026-09-13T10:52:49.433Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-672d787302a74fa2","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.441Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-91815acd4220217b76476efa: done: 0.538859 sec","job_status":"done","queue_duration_s":0.00381,"scheduling_latency_s":0.003875,"gitaly_calls":3,"gitaly_duration_s":0.184544,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.033779,"redis_read_bytes":239,"redis_write_bytes":21781,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002177,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002148,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.005079,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14466,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.018089,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000453,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000531,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.003125,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.014,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.018,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.276058,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.43870974,"duration_s":0.538859,"completed_at":"2026-09-13T10:52:49.983Z","load_balancing_strategy":"primary","db_duration_s":0.060243,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 445.190643] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:49.986Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"387fa4e3a9bf0a4396aed6f9","created_at":"2026-09-13T10:52:49.600Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.605Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-387fa4e3a9bf0a4396aed6f9: start","job_status":"start","queue_duration_s":0.381747,"scheduling_latency_s":0.381789} gitlab # [ 445.207316] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"387fa4e3a9bf0a4396aed6f9","created_at":"2026-09-13T10:52:49.600Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.605Z","job_size_bytes":23,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-387fa4e3a9bf0a4396aed6f9: done: 0.070101 sec","job_status":"done","queue_duration_s":0.381747,"scheduling_latency_s":0.381789,"redis_calls":7,"redis_duration_s":0.002164,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001614,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00055,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.070101,"completed_at":"2026-09-13T10:52:50.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011638,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 445.249568] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.058Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5f70864ef5550028c5ccdd78","created_at":"2026-09-13T10:52:49.787Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.793Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5f70864ef5550028c5ccdd78: start","job_status":"start","queue_duration_s":0.264962,"scheduling_latency_s":0.264991} gitlab # [ 445.266419] nixbot[3588]: {"ts": "2026-09-13T10:52:50.034Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 \"HTTP/1.1 201 Created\""} gitlab # [ 445.269291] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:50 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 237 gitlab # [ 445.406376] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.280Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5f70864ef5550028c5ccdd78","created_at":"2026-09-13T10:52:49.787Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.793Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5f70864ef5550028c5ccdd78: done: 0.22236 sec","job_status":"done","queue_duration_s":0.264962,"scheduling_latency_s":0.264991,"redis_calls":7,"redis_duration_s":0.003511,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00276,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000751,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.199895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.22236,"completed_at":"2026-09-13T10:52:50.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.494161] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.288Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"183504ab11033505442704d9","created_at":"2026-09-13T10:52:49.808Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.812Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-183504ab11033505442704d9: start","job_status":"start","queue_duration_s":0.475784,"scheduling_latency_s":0.475812} gitlab # [ 445.512775] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.354Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"183504ab11033505442704d9","created_at":"2026-09-13T10:52:49.808Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.812Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-183504ab11033505442704d9: done: 0.065948 sec","job_status":"done","queue_duration_s":0.475784,"scheduling_latency_s":0.475812,"redis_calls":7,"redis_duration_s":0.006482,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.065948,"completed_at":"2026-09-13T10:52:50.354Z","load_balancing_strategy":"primary","db_duration_s":0.010253,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses gitlab # [ 445.556986] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"772013046b4c8ab4e34ace9d","created_at":"2026-09-13T10:52:49.824Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.828Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-772013046b4c8ab4e34ace9d: start","job_status":"start","queue_duration_s":0.530541,"scheduling_latency_s":0.530588} gitlab # [ 445.571737] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.441Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"772013046b4c8ab4e34ace9d","created_at":"2026-09-13T10:52:49.824Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.828Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-772013046b4c8ab4e34ace9d: done: 0.082528 sec","job_status":"done","queue_duration_s":0.530541,"scheduling_latency_s":0.530588,"redis_calls":10,"redis_duration_s":0.006361,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003004,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00253,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000827,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.057457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.082528,"completed_at":"2026-09-13T10:52:50.441Z","load_balancing_strategy":"primary","db_duration_s":0.013125,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.614985] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.444Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2de21753d49d992c4bd4e0ed","created_at":"2026-09-13T10:52:49.829Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.832Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2de21753d49d992c4bd4e0ed: start","job_status":"start","queue_duration_s":0.611607,"scheduling_latency_s":0.61165} gitlab # [ 445.631689] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.502Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2de21753d49d992c4bd4e0ed","created_at":"2026-09-13T10:52:49.829Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.832Z","job_size_bytes":3,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2de21753d49d992c4bd4e0ed: done: 0.058878 sec","job_status":"done","queue_duration_s":0.611607,"scheduling_latency_s":0.61165,"redis_calls":7,"redis_duration_s":0.003555,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039906,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.058878,"completed_at":"2026-09-13T10:52:50.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.674280] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.504Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"63aa11f100a1a006abfd4210","created_at":"2026-09-13T10:52:49.833Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.836Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-63aa11f100a1a006abfd4210: start","job_status":"start","queue_duration_s":0.668273,"scheduling_latency_s":0.668307} gitlab # [ 445.690877] gitaly[891]: time="2026-09-13T10:52:50.564Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2D6BZD242JGJF82HAH04C0T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-13T10:52:50.550" grpc.time_ms=13.392 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 445.704514] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:50 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 98 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses, in 0.19 seconds) (finished: subtest: eval and build post commit statuses, in 11.29 seconds) (finished: run the VM test script, in 517.69 seconds) gitlab # [ 445.706594] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"63aa11f100a1a006abfd4210","created_at":"2026-09-13T10:52:49.833Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:cfc6cb7e9ac589948d38cba01bc2474abdf1c3f286e44c45fd4f7315244498f9","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.836Z","job_size_bytes":19,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-63aa11f100a1a006abfd4210: done: 0.052933 sec","job_status":"done","queue_duration_s":0.668273,"scheduling_latency_s":0.668307,"redis_calls":8,"redis_duration_s":0.004603,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003914,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000689,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027681,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.052933,"completed_at":"2026-09-13T10:52:50.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005015,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 445.749859] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.560Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b7ec8dedce555206b38261c4","created_at":"2026-09-13T10:52:49.838Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e6b1af094efc47a22f7a765ff4de932873f6cdeb518f26c41b5526619631b5bd","size_limiter":"validated","enqueued_at":"2026-09-13T10:52:49.840Z","job_size_bytes":66,"pid":3454,"sidekiq_tid":"m8ly","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b7ec8dedce555206b38261c4: start","job_status":"start","queue_duration_s":0.720026,"scheduling_latency_s":0.720063} gitlab # [ 445.768780] gitaly[891]: time="2026-09-13T10:52:50.569Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2D6BZDRE4Y80CQJVB2CT6MD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-13T10:53:20.467" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-13T10:52:50.567" grpc.time_ms=1.146 pid=891 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 445.783226] gitlab-workhorse[815]: gitlab 2001:db8:1::1 - - [2026/09/13:10:52:50 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 120 test script finished in 517.74s cleanup kill QemuMachine (pid 46) gitlab # [ 445.786829] sidekiq-cluster[3454]: {"severity":"INFO","time":"2026-09-13T10:52:50.603Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e2, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b7ec8dedce555206b38261c4","created_at":"2026-09-13T10:52:49.838Z","trace_propagation_headers":{"sentry-trace":"f62382e123e145bd98ccb25cc1e1660b-c38cfb621e4b431c","baggage":"sentry-trace_id=f62382e123e145bd98ccb25cc1e1660b,sentry-release=Unknown"},"correlation_id":"01M2D6BY2XNE6GDWJBX1J393X0","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) gitlab # " (finished: cleanup, in 1.62 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-13T10:51:10.943Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-13T10:52:39.060054Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-13T10:52:39.470Z', 'started_at': None, 'finished_at': '2026-09-13T10:52:49.412Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '3c4e47efc34bfd8c71a73a1cd8ddd548489eabe3', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-13T10:52:49.729Z', 'started_at': None, 'finished_at': '2026-09-13T10:52:49.988Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]