tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab: QEMU running (pid 46) gitlab # Formatting '/build/vm-state-gitlab/tmp.ZLvAlVX4GV', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) gitlab # Discarding device blocks: 0/262144 done gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab # Filesystem UUID: 320736cf-bfe5-451f-b574-3225322a0ee1 (finished: start all VMs, in 67.99 seconds) gitlab # Superblock backups stored on blocks: gitlab: waiting for unit setup-gitlab.service gitlab # 32768, 98304, 163840, 229376 gitlab: waiting for the VM to finish booting gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nm6l5xr71zy2ykiz51323bdr6i8ax6m6-nixos-system-gitlab-test/init regInfo=/nix/store/djrqp5nyvg78hzv0pj3rmb0qgyzp9ig8-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/djrqp5nyvg78hzv0pj3rmb0qgyzp9ig8-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000027] arm-pv: using stolen time PV gitlab # [ 0.000362] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000549] Console: colour dummy device 80x25 gitlab # [ 0.000556] printk: legacy console [tty0] enabled gitlab # [ 0.000729] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000735] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000814] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000942] landlock: Up and running. gitlab # [ 0.000945] Yama: becoming mindful. gitlab # [ 0.001350] LSM support for eBPF active gitlab # [ 0.001547] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001657] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002808] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003509] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003513] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003663] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004734] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004828] EFI services will not be available. gitlab # [ 0.004988] smp: Bringing up secondary CPUs ... gitlab # [ 0.005542] Detected PIPT I-cache on CPU1 gitlab # [ 0.005605] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.005676] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.005749] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.006613] Detected PIPT I-cache on CPU2 gitlab # [ 0.006665] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.006716] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.006769] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.007951] Detected PIPT I-cache on CPU3 gitlab # [ 0.008023] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.008095] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.008168] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.008514] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.008541] SMP: Total of 4 processors activated. gitlab # [ 0.008543] CPU: All CPU(s) started at EL1 gitlab # [ 0.008552] CPU features: detected: Branch Target Identification gitlab # [ 0.008556] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.008559] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.008562] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.008565] CPU features: detected: Common not Private translations gitlab # [ 0.008568] CPU features: detected: CRC32 instructions gitlab # [ 0.008570] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.008574] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.008577] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.008580] CPU features: detected: E0PD gitlab # [ 0.008582] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.008585] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.008588] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.008591] CPU features: detected: Fine Grained Traps gitlab # [ 0.008594] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.008597] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.008600] CPU features: detected: LSE atomic instructions gitlab # [ 0.008602] CPU features: detected: Privileged Access Never gitlab # [ 0.008605] CPU features: detected: PMUv3 gitlab # [ 0.008607] CPU features: detected: RAS Extension Support gitlab # [ 0.008610] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.008612] CPU features: detected: Random Number Generator gitlab # [ 0.008614] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.008617] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.008619] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.008623] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.008705] alternatives: applying system-wide alternatives gitlab # [ 0.011530] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.011748] Memory: 5963768K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.013294] devtmpfs: initialized gitlab # [ 0.016542] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.016614] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.016796] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.016800] 0 pages in range for non-PLT usage gitlab # [ 0.016802] 508288 pages in range for PLT usage gitlab # [ 0.016882] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.017664] DMI not present or invalid. gitlab # [ 0.020833] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.023185] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.023434] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.025508] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.025528] audit: initializing netlink subsys (disabled) gitlab # [ 0.025754] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.026226] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.026228] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.026232] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.026235] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.026238] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.026294] cpuidle: using governor ladder gitlab # [ 0.026334] cpuidle: using governor menu gitlab # [ 0.026577] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.026713] ASID allocator initialised with 65536 entries gitlab # [ 0.027886] Serial: AMBA PL011 UART driver gitlab # [ 0.033146] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.033428] printk: console [ttyAMA0] enabled gitlab # [ 0.038827] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.038839] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.038844] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.038849] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.038853] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.038857] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.038862] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.038866] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.046037] fbcon: Taking over console gitlab # [ 0.046048] ACPI: Interpreter disabled. gitlab # [ 0.047003] iommu: Default domain type: Translated gitlab # [ 0.047012] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.055917] SCSI subsystem initialized gitlab # [ 0.056266] usbcore: registered new interface driver usbfs gitlab # [ 0.056297] usbcore: registered new interface driver hub gitlab # [ 0.056321] usbcore: registered new device driver usb gitlab # [ 0.056592] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.056596] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.056604] PTP clock support registered gitlab # [ 0.056642] EDAC MC: Ver: 3.0.0 gitlab # [ 0.056950] scmi_core: SCMI protocol bus registered gitlab # [ 0.057595] FPGA manager framework gitlab # [ 0.058064] vgaarb: loaded gitlab # [ 0.060111] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.064915] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.064951] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.065460] netfs: FS-Cache loaded gitlab # [ 0.065609] pnp: PnP ACPI: disabled gitlab # [ 0.071546] NET: Registered PF_INET protocol family gitlab # [ 0.072982] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.114697] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.114766] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.114823] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.115243] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.116073] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.116279] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.116361] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.116538] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.116789] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.116818] NET: Registered PF_XDP protocol family gitlab # [ 0.116836] PCI: CLS 0 bytes, default 64 gitlab # [ 0.117053] Trying to unpack rootfs image as initramfs... gitlab # [ 0.121462] kvm [1]: HYP mode not available gitlab # [ 0.176801] Initialise system trusted keyrings gitlab # [ 0.177093] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.180429] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.184169] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.198968] Key type asymmetric registered gitlab # [ 0.198974] Asymmetric key parser 'x509' registered gitlab # [ 0.199023] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.199220] io scheduler mq-deadline registered gitlab # [ 0.199229] io scheduler kyber registered gitlab # [ 0.210610] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.212718] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.213266] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.213285] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.213298] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.213307] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.213330] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.213357] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.213451] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.213461] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.213467] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.213472] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.213477] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.213562] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.214344] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.214694] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.214713] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.214763] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.214783] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.215631] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.215978] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.216003] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.216052] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.216917] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.217268] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.217287] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.217329] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.218138] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.218491] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.218509] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.218551] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.219357] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.219696] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.219714] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.219757] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.220613] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.220963] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.220982] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.221030] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.221873] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.222231] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.222250] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.222291] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.222315] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.223160] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.223520] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.223569] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.224399] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.224770] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.224819] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.225523] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.225864] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.226297] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.226751] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.226770] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.226818] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.227636] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.228079] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.228097] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.228157] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.229230] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.229239] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.229245] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.229317] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.229388] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.229459] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.229531] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.229612] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.229683] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.229756] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.229839] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.229918] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.230019] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.230089] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.230117] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.230145] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.230180] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.230208] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.230236] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.230265] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.230294] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.230330] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.230364] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.230395] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.230431] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.230466] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.230494] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.230523] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.230551] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.230578] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.230606] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.230635] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.230664] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.230693] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.230727] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.230735] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.230740] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.233297] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.251699] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.255098] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.258405] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.261877] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.266197] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.269422] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.272786] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.276750] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.280580] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.283660] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.287156] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.296129] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.298772] msm_serial: driver initialized gitlab # [ 0.299059] SuperH (H)SCI(F) driver initialized gitlab # [ 0.299134] STM32 USART driver initialized gitlab # [ 0.325970] loop: module loaded gitlab # [ 0.326265] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.329398] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.332174] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.334798] virtio_blk virtio5: [vdb] 12395160 512-byte logical blocks (6.35 GB/5.91 GiB) gitlab # [ 0.337809] megasas: 07.734.00.00-rc1 gitlab # [ 0.338887] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.341964] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.341987] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.344876] Using buffer write method gitlab # [ 0.344920] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.347817] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.347858] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.351281] Using buffer write method gitlab # [ 0.351301] Concatenating MTD devices: gitlab # [ 0.351305] (0): "0.flash" gitlab # [ 0.351308] (1): "0.flash" gitlab # [ 0.351311] into device "0.flash" gitlab # [ 0.560531] Freeing initrd memory: 27184K gitlab # [ 0.570341] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.575342] thunder_xcv, ver 1.0 gitlab # [ 0.575424] thunder_bgx, ver 1.0 gitlab # [ 0.575456] nicpf, ver 1.0 gitlab # [ 0.576387] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.576394] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.576428] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.576436] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.576485] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.576490] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.576536] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.576542] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.576759] sky2: driver version 1.30 gitlab # [ 0.579371] usbcore: registered new interface driver usb-storage gitlab # [ 0.579478] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.579478] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.579499] usbserial: USB Serial support registered for generic gitlab # [ 0.579512] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.579832] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.580453] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.581874] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.581908] rtc-pl031 9010000.pl031: setting system clock to 2026-10-04T19:03:55 UTC (1791140635) gitlab # [ 0.582390] i2c_dev: i2c /dev entries driver gitlab # [ 0.586819] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.586827] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.587260] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.587848] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.588172] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.588618] hub 1-0:1.0: USB hub found gitlab # [ 0.588639] hub 1-0:1.0: 6 ports detected gitlab # [ 0.590411] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.590873] usbcore: registered new interface driver usbhid gitlab # [ 0.590880] usbhid: USB HID core driver gitlab # [ 0.610868] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.613299] drop_monitor: Initializing network drop monitor service gitlab # [ 0.613527] NET: Registered PF_INET6 protocol family gitlab # [ 0.614478] Segment Routing with IPv6 gitlab # [ 0.614496] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.614545] NET: Registered PF_PACKET protocol family gitlab # [ 0.614710] 9pnet: Installing 9P2000 support gitlab # [ 0.617114] Key type dns_resolver registered gitlab # [ 0.627419] registered taskstats version 1 gitlab # [ 0.627760] Loading compiled-in X.509 certificates gitlab # [ 0.643052] Demotion targets for Node 0: null gitlab # [ 0.643256] Key type .fscrypt registered gitlab # [ 0.643262] Key type fscrypt-provisioning registered gitlab # [ 0.643387] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.643407] ima: Allocated hash algorithm: sha1 gitlab # [ 0.643431] ima: No architecture policies found gitlab # [ 0.644312] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.665365] clk: Disabling unused clocks gitlab # [ 0.665390] PM: genpd: Disabling unused power domains gitlab # [ 0.669162] Freeing unused kernel memory: 4736K gitlab # [ 0.669345] Run /init as init process gitlab # [ 0.695456] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.836192] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 0.983688] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.029591] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.029627] systemd[1]: Detected virtualization qemu. gitlab # [ 1.029668] systemd[1]: Detected architecture arm64. gitlab # [ 1.029676] systemd[1]: Running in initrd. gitlab # [ 1.030109] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.030246] systemd[1]: Hostname set to . gitlab # [ 1.112155] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.212323] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.232212] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.277444] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.291055] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.291315] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.291350] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.291363] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.291390] systemd[1]: Reached target Path Units. gitlab # [ 1.291409] systemd[1]: Reached target Slice Units. gitlab # [ 1.291427] systemd[1]: Reached target Swaps. gitlab # [ 1.291445] systemd[1]: Reached target Timer Units. gitlab # [ 1.291615] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.291781] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.291949] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.292095] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.292353] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.292376] systemd[1]: Reached target Socket Units. gitlab # [ 1.294008] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.295743] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.295814] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.297564] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.316317] systemd[1]: Starting Journal Service... gitlab # [ 1.318658] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.318797] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.320547] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.326101] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.329167] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.329701] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.333052] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.336720] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.344898] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.345120] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.345124] [drm] features: -context_init gitlab # [ 1.346048] [drm] number of scanouts: 1 gitlab # [ 1.346061] [drm] number of cap sets: 0 gitlab # [ 1.347913] systemd-journald[96]: Collecting audit messages is disabled. gitlab # [ 1.356197] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.356581] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.359591] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.359602] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.364220] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.365824] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.369204] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.369931] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.386034] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.386327] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.396489] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.398126] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.407096] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.407313] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.407350] systemd[1]: Reached target Local File Systems. gitlab # [ 1.410942] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.420409] systemd[1]: Started Journal Service. gitlab # [ 1.416280] systemd-modules-load[97]: Using 4 probe threads gitlab # [ 1.419105] systemd-modules-load[97]: Module 'virtio_balloon' is built in gitlab # [ 1.420369] systemd-modules-load[97]: Module 'virtio_console' is built in gitlab # [ 1.421591] systemd-modules-load[97]: Module 'virtio_rng' is built in gitlab # [ 1.422818] systemd-modules-load[97]: Inserted module 'dm_mod' gitlab # [ 1.423919] systemd-modules-load[97]: Inserted module 'virtio_gpu' gitlab # [ 1.425099] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.426563] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.440075] systemd-udevd[108]: Using default interface naming scheme 'v261'. gitlab # [ 1.442161] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.453655] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.473166] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.508658] systemd-vconsole-setup[131]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.512428] systemd[1]: Finished Virtual Console Setup. gitlab # [ 1.948134] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 1.949523] systemd[1]: Reached target System Initialization. gitlab # [ 1.950831] systemd[1]: Reached target Basic System. gitlab # [ 2.075456] (udev-worker)[120]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.078997] (udev-worker)[120]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.080725] (udev-worker)[124]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.101107] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.105338] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.110492] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.112896] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.147337] systemd-fsck[154]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.150139] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.319252] systemd[1]: Mounting /sysroot... gitlab # [ 2.357488] EXT4-fs (vda): mounted filesystem 320736cf-bfe5-451f-b574-3225322a0ee1 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.355081] systemd[1]: Mounted /sysroot. gitlab # [ 2.356713] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.359001] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.360281] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.366256] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.382925] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.387893] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.390515] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.393648] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.399350] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.401640] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.411911] systemd-sysroot-fstab-check[167]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.419951] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.427168] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.428232] systemd[1]: Reload requested from client PID 167 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.429759] systemd[1]: Reloading... gitlab # [ 2.644709] (udev-worker)[142]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 2.647973] (udev-worker)[142]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 2.658028] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.709752] systemd[1]: Reloading finished in 281 ms. gitlab # [ 2.760705] systemd-sysroot-fstab-check[167]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.763742] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.765060] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.766243] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.769520] systemd-sysroot-fstab-check[167]: Requesting swap.target/start/replace... gitlab # [ 2.777790] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 2.779428] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 2.781036] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 2.788187] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.789565] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 2.791202] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 2.792454] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 2.793700] systemd[1]: Starting Virtual Console Setup... gitlab # [ 2.819374] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 2.821814] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 2.832943] systemd-vconsole-setup[200]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 2.835626] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.322197] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.365415] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.366698] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.368001] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.369617] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.397092] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.399224] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.410370] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.411579] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.414003] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.431966] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.433334] systemd[1]: Stopped target Basic System. gitlab # [ 3.434496] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.435746] systemd[1]: Stopped target Path Units. gitlab # [ 3.436955] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.438654] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.440421] systemd[1]: Stopped target Slice Units. gitlab # [ 3.441550] systemd[1]: Stopped target Socket Units. gitlab # [ 3.444622] systemd[1]: Stopped target System Initialization. gitlab # [ 3.445975] systemd[1]: Stopped target Swaps. gitlab # [ 3.447224] systemd[1]: Stopped target Timer Units. gitlab # [ 3.451358] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.453581] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.457023] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.458980] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.460223] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.461652] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.463017] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.464721] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.465912] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.467543] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.468770] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.470522] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.472422] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.474042] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.475458] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.476765] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.478230] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.479773] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.481029] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.482655] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.484115] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.485152] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.486565] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.487704] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.489013] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.490276] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.491657] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.492788] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.493978] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.495256] systemd[1]: systemd-udevd.service: Consumed 2.022s CPU time over 2.074s wall clock time, 32.4M memory peak. gitlab # [ 3.497161] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.498474] systemd[1]: Closed udev Control Socket. gitlab # [ 3.499373] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.500388] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.501734] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.502891] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.504550] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.505851] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.507146] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.557561] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.559295] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.560540] systemd[1]: Reached target Switch Root. gitlab # [ 3.561668] systemd[1]: Starting NixOS Activation... gitlab # [ 3.679029] initrd-nixos-activation-start[226]: booting system configuration /nix/store/nm6l5xr71zy2ykiz51323bdr6i8ax6m6-nixos-system-gitlab-test gitlab # [ 3.719443] initrd-nixos-activation-start[226]: running activation script... gitlab # [ 3.936823] initrd-nixos-activation-start[249]: setting up /etc... gitlab # [ 4.036212] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.038062] systemd[1]: Finished NixOS Activation. gitlab # [ 4.039206] systemd[1]: Starting Switch Root... gitlab # [ 4.076119] systemd[1]: Switching root. gitlab # [ 4.231241] systemd-journald[96]: Received SIGTERM from PID 1 (systemd). gitlab # [ 4.804087] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 4.804196] systemd[1]: Detected virtualization qemu. gitlab # [ 4.804259] systemd[1]: Detected architecture arm64. gitlab # [ 4.804358] systemd[1]: Detected first boot. gitlab # [ 4.805104] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.112722] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.191693] systemd[1]: Applying preset policy. gitlab # [ 5.392336] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 5.538157] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 5.538969] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 5.540974] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 5.544410] systemd[1]: Created slice Slice /system/getty. gitlab # [ 5.546240] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 5.548545] systemd[1]: Created slice User and Session Slice. gitlab # [ 5.549139] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 5.549623] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 5.550034] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 5.550540] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 5.550998] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 5.551438] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 5.551870] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 5.552465] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 5.552878] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 5.553318] systemd[1]: Reached target Path Units. gitlab # [ 5.553756] systemd[1]: Reached target Remote File Systems. gitlab # [ 5.554186] systemd[1]: Reached target Slice Units. gitlab # [ 5.554638] systemd[1]: Reached target Swaps. gitlab # [ 5.556070] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 5.557608] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 5.558734] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 5.559986] systemd[1]: Listening on Factory Reset Management. gitlab # [ 5.560502] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 5.564839] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 5.565913] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 5.567724] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 5.568421] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 5.568864] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.569263] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 5.571595] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 5.572165] systemd[1]: Listening on udev Control Socket. gitlab # [ 5.572705] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 5.598835] systemd[1]: Mounting Huge Pages File System... gitlab # [ 5.601744] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 5.605054] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 5.608676] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 5.612255] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 5.615379] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 5.616784] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 5.631659] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 5.632189] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 5.632610] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 5.641395] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 5.641968] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 5.648433] systemd[1]: Starting Journal Service... gitlab # [ 5.652261] systemd[1]: Starting Load Kernel Modules... gitlab # [ 5.656256] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 5.672011] fuse: init (API version 7.45) gitlab # [ 5.672157] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 5.672799] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.676004] systemd-journald[320]: Collecting audit messages is enabled. gitlab # [ 5.682488] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 5.689302] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 5.690130] systemd[1]: Mounted Huge Pages File System. gitlab # [ 5.690949] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 5.691715] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 5.692487] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 5.693787] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 5.694877] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 5.695821] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 5.696649] systemd[1]: Started Journal Service. gitlab # [ 5.690625] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 5.697703] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 5.698882] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 5.700404] systemd-modules-load[321]: Using 3 probe threads gitlab # [ 5.701740] systemd-modules-load[321]: Module 'atkbd' is built in gitlab # [ 5.702840] systemd-modules-load[321]: Module 'loop' is built in gitlab # [ 5.703967] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 5.705116] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 5.706314] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 5.707269] systemd[1]: Mounting FUSE Control File System... gitlab # [ 5.708277] systemd-modules-load[321]: Inserted module 'tls'[ 5.712208] EXT4-fs (vda): re-mounted 320736cf-bfe5-451f-b574-3225322a0ee1. gitlab # gitlab # [ 5.710597] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 5.712427] systemd[1]: Finished Load Kernel Modules. gitlab # [ 5.726818] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 5.728186] systemd[1]: Mounted FUSE Control File System. gitlab # [ 5.731416] systemd-oomd[322]: No swap; memory pressure usage will be degraded gitlab # [ 5.733821] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 5.735626] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 5.740220] systemd[1]: Starting Firewall... gitlab # [ 5.742123] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 5.744212] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 5.748187] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 5.751676] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 5.756291] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 5.759435] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 5.765513] systemd-journald[320]: Received client request to flush runtime journal. gitlab # [ 5.788834] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 5.790059] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 5.790989] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 5.792297] systemd[1]: Reached target First Boot Complete. gitlab # [ 5.793179] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 5.794114] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 5.795214] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 5.796326] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 5.839844] systemd-udevd[354]: Using default interface naming scheme 'v261'. gitlab # [ 5.868571] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.054049] systemd[1]: Finished Firewall. gitlab # [ 6.295362] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 6.354919] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.398118] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 6.426386] (udev-worker)[390]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 6.429885] (udev-worker)[390]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.431880] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 6.434121] (udev-worker)[470]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 6.445294] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 6.446959] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.449152] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.453528] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.455681] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.457666] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.520755] mousedev: PS/2 mouse device common for all mice gitlab # [ 6.536117] systemd[1]: Mounting /run/wrappers... gitlab # [ 6.563434] systemd[1]: Mounted /run/wrappers. gitlab # [ 6.564941] systemd[1]: Reached target Local File Systems. gitlab # [ 6.566643] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 6.567930] systemd[1]: Starting register-nix-paths.service... gitlab # [ 6.570129] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 6.571489] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 6.575554] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 6.592378] systemd[1]: Starting Create System Files and Directories... gitlab # [ 6.614902] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 6.616193] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 6.632356] systemd[1]: Finished Create System Files and Directories. gitlab # [ 6.637715] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 6.644310] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 6.669784] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 6.679676] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 6.683030] systemd[1]: Starting Update is Completed... gitlab # [ 6.696962] systemd[1]: Finished Update is Completed. gitlab # [ 6.765422] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 6.767536] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 6.769409] systemd[1]: suid-sgid-wrappers.service: Consumed 132ms CPU time over 196ms wall clock time, 6.2M memory peak, 1.9M read from disk. gitlab # [ 6.955599] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 7.023208] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.025439] systemd[1]: Reached target System Initialization. gitlab # [ 7.028909] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.030549] systemd[1]: Started logrotate.timer. gitlab # [ 7.031356] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.033203] systemd[1]: Reached target Timer Units. gitlab # [ 7.033993] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.034961] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.042534] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.044815] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.048495] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.049602] systemd[1]: Reached target Socket Units. gitlab # [ 7.051982] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.070230] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.077510] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 7.079311] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 7.086494] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.090604] dbus-broker-launch[565]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.091984] dbus-broker-launch[565]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.093216] dbus-broker-launch[565]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.097719] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.106721] dbus-broker-launch[565]: Ready gitlab # [ 7.107686] systemd[1]: Reached target Basic System. gitlab # [ 7.109349] systemd[1]: Started backdoor.service. gitlab # [ 7.112254] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.115328] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.119137] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.124261] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.128323] systemd[1]: Starting Post-Boot Actions... gitlab # connecting to host... gitlab # [ 7.136599] logrotate[577]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.138731] logrotate[577]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 7.143785] logrotate[577]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.145281] logrotate[577]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.146346] logrotate[577]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.147429] logrotate[577]: Allocating hash table for state file, size 64 entries gitlab # [ 7.149603] logrotate[577]: Handling 4 logs gitlab # [ 7.150251] logrotate[577]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.152510] logrotate[577]: considering log /var/log/btmp gitlab # [ 7.153259] logrotate[577]: Creating new state gitlab # [ 7.153887] logrotate[577]: Now: 2026-10-04 19:04 gitlab # [ 7.154697] logrotate[577]: Last rotated at 2026-10-04 19:00 gitlab # [ 7.155591] logrotate[577]: log does not need rotating (log has already been rotated) gitlab # [ 7.156780] logrotate[577]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.159096] logrotate[577]: considering log /var/log/wtmp gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.163253] logrotate[577]: Creating new state gitlab # [ 7.163953] logrotate[577]: Now: 2026-10-04 19:04 gitlab: connected to guest root shell gitlab # [ 7.164722] logrotate[577]: Last rotated at 2026-10-04 19:00 gitlab: (connecting took 7.60 seconds) gitlab # [ 7.165490] logrotate[577]: log does not need rotating (log has already been rotated) gitlab: (finished: waiting for the VM to finish booting, in 7.60 seconds) gitlab # [ 7.166616] logrotate[577]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.169220] logrotate[577]: switching euid from 0 to 165 and egid from 0 to 165 (pid 577) gitlab # [ 7.170318] logrotate[577]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.171151] logrotate[577]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.172215] logrotate[577]: Creating new state gitlab # [ 7.172795] logrotate[577]: switching euid from 165 to 0 and egid from 165 to 0 (pid 577) gitlab # [ 7.173829] logrotate[577]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.175419] logrotate[577]: switching euid from 0 to 60 and egid from 0 to 60 (pid 577) gitlab # [ 7.176658] logrotate[577]: considering log /var/log/nginx/*.log gitlab # [ 7.177449] logrotate[577]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.178455] logrotate[577]: Creating new state gitlab # [ 7.179038] logrotate[577]: not running postrotate script, since no logs were rotated gitlab # [ 7.180716] logrotate[577]: switching euid from 60 to 0 and egid from 60 to 0 (pid 577) gitlab # [ 7.182038] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.183461] nsncd[578]: Oct 04 19:04:02.097 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.187030] systemd[1]: Starting resolvconf update... gitlab # [ 7.189101] systemd-vconsole-setup[571]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 7.191340] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.193031] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.194744] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.195598] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 7.196711] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.197620] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.198590] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.199570] systemd[1]: Starting User Login Management... gitlab # [ 7.230544] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 7.231655] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 7.232662] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 7.233538] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 7.234538] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 7.235460] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 7.237803] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 7.238715] systemd[1]: nscd.service: Consumed 28ms CPU time over 110ms wall clock time, 7.7M memory peak, 5.8M read from disk. gitlab # [ 7.241517] systemd-logind[616]: New seat seat0. gitlab # [ 7.245125] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.247535] systemd-logind[616]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 7.249905] systemd-logind[616]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 7.252390] systemd-logind[616]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 7.253565] systemd[1]: Started User Login Management. gitlab # [ 7.267962] systemd[1]: Starting linger-users.service... gitlab # [ 7.272757] systemd[1]: Finished resolvconf update. gitlab # [ 7.273874] systemd[1]: Reached target Preparation for Network. gitlab # [ 7.280461] nsncd[668]: Oct 04 19:04:02.200 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.282653] systemd[1]: Starting DHCP Client... gitlab # [ 7.284826] systemd[1]: Starting Address configuration of eth1... gitlab # [ 7.290183] systemd[1]: Starting Extra networking commands.... gitlab # [ 7.291120] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.293710] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 7.294857] systemd[1]: Finished linger-users.service. gitlab # [ 7.299367] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.301218] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.313257] network-addresses-eth1-start[695]: adding address 192.168.1.1/24... done gitlab # [ 7.320744] network-addresses-eth1-start[695]: adding address 2001:db8:1::1/64... done gitlab # [ 7.328354] systemd[1]: Finished Address configuration of eth1. gitlab # [ 7.365809] dhcpcd[710]: dhcpcd-10.3.2 starting gitlab # [ 7.368255] dhcpcd[760]: dev: loaded udev gitlab # [ 7.368947] systemd[1]: Finished Extra networking commands.. gitlab # [ 7.369798] systemd[1]: Reached target Network. gitlab # [ 7.372223] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 7.377242] systemd[1]: Starting Nginx Web Server... gitlab # [ 7.382649] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 7.383056] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 7.383917] systemd[1]: Starting PostgreSQL Server... gitlab # [ 7.390179] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 7.392866] systemd[1]: Starting Permit User Sessions... gitlab # [ 7.400971] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 7.426448] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 7.425934] systemd[1]: Finished Permit User Sessions. gitlab # [ 7.436197] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 7.436696] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 7.438694] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 7.439015] cfg80211: failed to load regulatory.db gitlab # [ 7.439892] systemd[1]: Started Getty on tty1. gitlab # [ 7.441278] systemd[1]: Reached target Login Prompts. gitlab # [ 7.478147] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 7.476215] dhcpcd[760]: eth0: waiting for carrier gitlab # [ 7.478378] dhcpcd[760]: eth0: carrier acquired gitlab # [ 7.482538] dhcpcd[760]: DUID 00:01:00:01:32:55:5f:a2:52:54:00:12:34:56 gitlab # [ 7.483973] dhcpcd[760]: eth0: IAID 00:12:34:56 gitlab # [ 7.484970] dhcpcd[760]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 7.539954] nginx-pre-start[799]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 7.543184] nginx-pre-start[799]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 7.548093] systemd[1]: Started Nginx Web Server. gitlab # [ 7.568856] postgresql-pre-start[798]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 7.570991] postgresql-pre-start[798]: This user must also own the server process. gitlab # [ 7.573263] postgresql-pre-start[798]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 7.574905] postgresql-pre-start[798]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 7.577822] postgresql-pre-start[798]: The default text search configuration will be set to "english". gitlab # [ 7.579201] postgresql-pre-start[798]: Data page checksums are enabled. gitlab # [ 7.580869] postgresql-pre-start[798]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 7.584498] redis-server[794]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 7.586700] postgresql-pre-start[798]: creating subdirectories ... ok gitlab # [ 7.587636] postgresql-pre-start[798]: selecting dynamic shared memory implementation ... posix gitlab # [ 7.589378] redis-server[794]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 7.590754] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 7.591656] redis-server[794]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=794, just started gitlab # [ 7.593396] redis-server[794]: Configuration loaded gitlab # [ 7.594236] redis[794]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 7.595221] redis[794]: Running mode=standalone, port=0. gitlab # [ 7.596164] redis[794]: Server initialized gitlab # [ 7.596899] redis[794]: Ready to accept connections unix gitlab # [ 7.597759] redis[794]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 7.620811] postgresql-pre-start[798]: selecting default "max_connections" ... 100 gitlab # [ 7.669638] postgresql-pre-start[798]: selecting default "shared_buffers" ... 128MB gitlab # [ 7.954580] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 8.122287] postgresql-pre-start[798]: selecting default time zone ... UTC gitlab # [ 8.124217] postgresql-pre-start[798]: creating configuration files ... ok gitlab # [ 8.200274] gitlab-workhorse[822]: time="2026-10-04T19:04:03Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 8.202979] gitlab-workhorse[822]: time="2026-10-04T19:04:03Z" level=info msg="Using redis/go-redis" gitlab # [ 8.207407] gitlab-workhorse[822]: time="2026-10-04T19:04:03Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 8.369036] postgresql-pre-start[798]: running bootstrap script ... ok gitlab # [ 9.023871] postgresql-pre-start[798]: performing post-bootstrap initialization ... ok gitlab # [ 9.176480] postgresql-pre-start[798]: syncing data to disk ... ok gitlab # [ 9.178069] postgresql-pre-start[798]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 9.180303] postgresql-pre-start[798]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 9.183668] postgresql-pre-start[798]: Success. You can now start the database server using: gitlab # [ 9.185630] postgresql-pre-start[798]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 9.286085] dhcpcd[760]: eth0: soliciting a DHCP lease gitlab # [ 9.302129] postgres[848]: [848] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 9.304677] dhcpcd[760]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.306833] postgres[848]: [848] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 9.308217] postgres[848]: [848] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 9.309717] postgres[848]: [848] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 9.316065] postgres[866]: [866] LOG: database system was shut down at 2026-10-04 19:04:03 GMT gitlab # [ 9.321511] postgres[848]: [848] LOG: database system is ready to accept connections gitlab # [ 9.324634] systemd[1]: Started PostgreSQL Server. gitlab # [ 9.325865] dhcpcd[760]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.330491] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 9.365393] dhcpcd[760]: eth0: soliciting an IPv6 router gitlab # [ 9.366871] dhcpcd[760]: eth0: Router Advertisement from fe80::2 gitlab # [ 9.368367] dhcpcd[760]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 9.369806] dhcpcd[760]: eth0: adding route to fec0::/64 gitlab # [ 9.371098] dhcpcd[760]: eth0: adding default route via fe80::2 gitlab # [ 9.557151] postgresql-setup-start[877]: CREATE DATABASE gitlab # [ 9.591911] postgresql-setup-start[882]: CREATE ROLE gitlab # [ 9.607727] postgresql-setup-start[884]: ALTER DATABASE gitlab # [ 9.638654] postgresql-setup-start[889]: CREATE ROLE gitlab # [ 9.641105] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 9.643044] systemd[1]: Reached target PostgreSQL. gitlab # [ 9.645715] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 9.790980] gitlab-postgresql-start[903]: CREATE DATABASE gitlab # [ 9.804557] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[895]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 9.806902] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[895]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 9.832513] gitlab-postgresql-start[910]: CREATE EXTENSION gitlab # [ 9.864741] systemd[1]: Finished gitlab-config.service. gitlab # [ 9.871343] systemd[1]: Started gitaly.service. gitlab # [ 9.873237] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 9.936873] gitlab-postgresql-start[916]: CREATE EXTENSION gitlab # [ 9.945170] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 10.438984] gitaly[923]: time="2026-10-04T19:04:05.360Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=923 gitlab # [ 10.452533] gitaly[923]: time="2026-10-04T19:04:05.360Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=923 gitlab # [ 10.455148] gitaly[923]: time="2026-10-04T19:04:05.360Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=923 gitlab # [ 10.458484] gitaly[923]: time="2026-10-04T19:04:05.360Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=923 gitlab # [ 10.461311] gitaly[923]: time="2026-10-04T19:04:05.360Z" level=info msg="Starting Gitaly" pid=923 version= gitlab # [ 10.658840] gitaly[923]: time="2026-10-04T19:04:05.580Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=219 pid=923 gitlab # [ 10.661095] gitaly[923]: time="2026-10-04T19:04:05.581Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=923 gitlab # [ 10.662990] gitaly[923]: time="2026-10-04T19:04:05.582Z" level=info msg="finished initializing command factory" duration_ms=0 pid=923 gitlab # [ 10.665052] gitaly[923]: time="2026-10-04T19:04:05.582Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-923/git-exec-1494762958.d/git pid=923 gitlab # [ 10.667399] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="finished detecting git version" duration_ms=5 pid=923 gitlab # [ 10.669490] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="using Git version" pid=923 version=2.53.g gitlab # [ 10.671217] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="clearing disk cache object folder" pid=923 storage=default gitlab # [ 10.673302] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3135104438 pid=923 storage=default gitlab # [ 10.676500] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3135104438 pid=923 storage=default gitlab # [ 10.680481] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=923 gitlab # [ 10.683703] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=923 gitlab # [ 10.687065] gitaly[923]: time="2026-10-04T19:04:05.587Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache3135104438 clear_duration_ms=0 pid=923 storage=default gitlab # [ 10.690255] gitaly[923]: time="2026-10-04T19:04:05.600Z" level=info msg="counting repositories" pid=923 gitlab # [ 10.691521] gitaly[923]: time="2026-10-04T19:04:05.601Z" level=info msg="starting to count repositories" pid=923 storage_path=/var/gitlab/state/repositories gitlab # [ 10.693419] gitaly[923]: time="2026-10-04T19:04:05.600Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=923 gitlab # [ 10.696138] gitaly[923]: time="2026-10-04T19:04:05.601Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=923 schema=unix gitlab # [ 10.698570] gitaly[923]: time="2026-10-04T19:04:05.601Z" level=info msg="listening at address" address=/run/gitaly/gitaly-923/sock.d/intern pid=923 schema=unix gitlab # [ 10.700912] gitaly[923]: time="2026-10-04T19:04:05.601Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=923 storage_path=/var/gitlab/state/repositories gitlab # [ 10.703644] gitaly[923]: time="2026-10-04T19:04:05.601Z" level=info msg="completed counting all repositories" counting_duration=0 pid=923 gitlab # [ 10.705965] gitaly[923]: time="2026-10-04T19:04:05.600Z" level=info msg="finished tempdir cleaner walk" pid=923 storage=default time_ms=0 gitlab # [ 10.708068] gitaly[923]: time="2026-10-04T19:04:05.602Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=923 gitlab # [ 10.710128] gitaly[923]: time="2026-10-04T19:04:05.602Z" level=info msg="Started Gitaly" duration_ms=241 pid=923 gitlab # [ 10.711868] gitaly[923]: time="2026-10-04T19:04:05.603Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=923 gitlab # [ 10.714108] gitaly[923]: time="2026-10-04T19:04:05.603Z" level=info msg="maintenance: daily scheduled" pid=923 scheduled="2026-10-05 12:00:00 +0000 UTC" gitlab # [ 10.955265] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.960169] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 10.968352] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.157086] gitaly[923]: time="2026-10-04T19:04:09.078Z" level=info msg="License database preloaded" duration_ms=3717 pid=923 gitlab # [ 14.296613] dhcpcd[760]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.298178] dhcpcd[760]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.299475] dhcpcd[760]: eth0: adding default route via 10.0.2.2 gitlab # [ 14.490387] systemd[1]: Started DHCP Client. gitlab # [ 14.493586] systemd[1]: Reached target Network is Online. gitlab # [ 56.883249] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[926]: W, [2026-10-04T19:04:51.804437 #926] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 58.267543] gitlab-workhorse[822]: time="2026-10-04T19:04:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 59.889529] postgres[1462]: [1462] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 59.889797] postgres[1462]: [1462] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 59.889866] postgres[1462]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 59.889927] postgres[1462]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 59.889989] postgres[1462]: attidentity AS identity, gitlab # [ 59.890049] postgres[1462]: attgenerated as attgenerated gitlab # [ 59.890109] postgres[1462]: FROM pg_attribute a gitlab # [ 59.890189] postgres[1462]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 59.890244] postgres[1462]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 59.890301] postgres[1462]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 59.890365] postgres[1462]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 59.890426] postgres[1462]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 59.890483] postgres[1462]: ORDER BY a.attnum gitlab # [ 59.890543] postgres[1462]: gitlab # [ 65.040294] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1513]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 65.040573] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1513]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 68.278128] gitlab-workhorse[822]: time="2026-10-04T19:05:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 78.288242] gitlab-workhorse[822]: time="2026-10-04T19:05:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 86.437166] postgres[1607]: [1607] ERROR: relation "table_without_models" does not exist gitlab # [ 86.437676] postgres[1607]: [1607] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 88.297988] gitlab-workhorse[822]: time="2026-10-04T19:05:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 98.298881] gitlab-workhorse[822]: time="2026-10-04T19:05:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 104.173089] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 104.176087] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 104.178591] systemd[1]: Starting gitlab.service... gitlab # [ 104.707421] bundle[1855]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 104.707705] bundle[1855]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 104.712097] bundle[1855]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 108.305359] gitlab-workhorse[822]: time="2026-10-04T19:05:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.406914] sidekiq-cluster[1854]: {"severity":"INFO","time":"2026-10-04T19:05:44.299Z","message":"Starting cluster with 1 processes"} gitlab # [ 109.537340] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.427Z","pid":1855,"message":"* Enabling systemd notification integration"} gitlab # [ 109.539990] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.458Z","pid":1855,"message":"Puma starting in cluster mode..."} gitlab # [ 109.542369] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.458Z","pid":1855,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 109.544894] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.458Z","pid":1855,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 109.547920] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Min threads: 0"} gitlab # [ 109.550046] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Max threads: 4"} gitlab # [ 109.552416] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Environment: production"} gitlab # [ 109.554590] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Master PID: 1855"} gitlab # [ 109.556698] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Workers: 2"} gitlab # [ 109.558666] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 109.561386] bundle[1855]: {"timestamp":"2026-10-04T19:05:44.459Z","pid":1855,"message":"* Preloading application"} gitlab # [ 118.315429] gitlab-workhorse[822]: time="2026-10-04T19:05:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 128.320769] gitlab-workhorse[822]: time="2026-10-04T19:06:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 138.319153] gitlab-workhorse[822]: time="2026-10-04T19:06:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 148.333422] gitlab-workhorse[822]: time="2026-10-04T19:06:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 154.711754] bundle[1855]: W, [2026-10-04T19:06:29.632848 #1855] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 158.341701] gitlab-workhorse[822]: time="2026-10-04T19:06:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 161.100461] sidekiq-cluster[1904]: 2026-10-04T19:06:36.021Z pid=1904 tid=wrc INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1904, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>425517056} gitlab # [ 161.410182] sidekiq-cluster[1904]: W, [2026-10-04T19:06:36.332117 #1904] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 161.561364] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:36.483Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 164.989115] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.910Z","message":"GitLab reliable fetch activated!"} gitlab # [ 164.989430] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.910Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 165.024367] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.946Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 165.028088] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.949Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 165.031750] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.952Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 165.035413] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.954Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 165.039373] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.956Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 165.042921] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.959Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 165.046548] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.961Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 165.049846] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.963Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 165.053161] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.964Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 165.056491] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.966Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 165.060250] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.968Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 165.063758] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.971Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 165.067223] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.972Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 165.070844] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.974Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 165.074355] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.976Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 165.077720] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.979Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 165.080726] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.982Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 165.083501] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.985Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 165.086677] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.987Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 165.089643] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.990Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 165.092716] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.993Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 165.095757] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.995Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 165.098581] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:39.997Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 165.101300] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.000Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 165.103875] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.002Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 165.106416] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.005Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 165.108751] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.008Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 165.111043] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.011Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 165.113491] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.013Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 165.115874] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.016Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 165.118177] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.019Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 165.120595] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.022Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 165.122754] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.025Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 165.125053] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.027Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 165.127241] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.030Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 165.129505] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.032Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 165.131677] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.035Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 165.133880] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.037Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 165.135988] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.039Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 165.138270] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.041Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 165.140580] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.044Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 165.142796] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.045Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 165.145143] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.048Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 165.147246] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.050Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 165.149597] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.053Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 165.151904] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.056Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 165.154289] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.058Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 165.156528] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.060Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 165.158673] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.063Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 165.160982] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.065Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 165.163135] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.068Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 165.165368] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.071Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 165.167517] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.084Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 165.170099] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.087Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 165.172353] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.090Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 165.174558] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.093Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 165.176846] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.095Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 165.179169] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.098Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 165.181660] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.100Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 165.271059] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.193Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 165.276637] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.198Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 165.279965] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.200Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 165.283702] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.205Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 165.287418] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.209Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 165.291224] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.211Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 165.294886] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.215Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 165.298416] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.217Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 165.301972] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.220Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 165.305692] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.222Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 165.309390] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.226Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 165.313103] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.228Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 165.316650] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.230Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 165.320257] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.233Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 165.323921] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.235Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 165.327515] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.239Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 165.330963] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.241Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 165.334295] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.244Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 165.337683] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.246Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 165.340551] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.250Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 165.343232] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.252Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 165.346313] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.254Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 165.349268] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.257Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 165.351938] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.261Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 165.354621] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.263Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 165.357321] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.265Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 165.359709] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.269Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 165.362388] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.272Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 165.364885] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.275Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 165.367323] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.277Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 165.369810] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.280Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 165.372165] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.282Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 165.374509] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.285Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 165.376950] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.287Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 165.379428] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.291Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 165.381598] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.294Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 165.383999] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.296Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 165.386408] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.300Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 165.389027] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.302Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 165.393967] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.315Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 165.396456] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.318Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 165.400057] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.322Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 165.404031] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.326Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 165.407799] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.329Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 165.410350] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.332Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 165.413527] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.335Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 165.416028] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.337Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 165.418357] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.339Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 165.420827] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.342Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 165.423170] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:06:40.345Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 168.349532] gitlab-workhorse[822]: time="2026-10-04T19:06:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 178.358517] gitlab-workhorse[822]: time="2026-10-04T19:06:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 188.366338] gitlab-workhorse[822]: time="2026-10-04T19:07:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 198.369673] gitlab-workhorse[822]: time="2026-10-04T19:07:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 208.377520] gitlab-workhorse[822]: time="2026-10-04T19:07:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 213.312655] postgres[2813]: [2813] ERROR: relation "table_without_models" does not exist gitlab # [ 213.314573] postgres[2813]: [2813] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 217.325878] postgres[2851]: [2851] ERROR: relation "table_without_models" does not exist gitlab # [ 217.326099] postgres[2851]: [2851] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 218.217594] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:33.138Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 218.223482] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:33.145Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 218.225686] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:33.146Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 218.227483] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:33.146Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 218.229658] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:33.148Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 218.387000] gitlab-workhorse[822]: time="2026-10-04T19:07:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 221.540235] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:36.461Z","message":"Cleaning working queues"} gitlab # [ 221.542561] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:36.464Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1b05a32811a3ce57d27f034f","created_at":"2026-10-04T19:05:38.563Z","trace_propagation_headers":{"sentry-trace":"07b54df8b35141258dc792dc046ae114-a338b19e565d4cb6","baggage":"sentry-trace_id=07b54df8b35141258dc792dc046ae114,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"972d644ddec575a5ea7fa645b2e6543f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T19:05:38.574Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1b05a32811a3ce57d27f034f: start","job_status":"start","queue_duration_s":117.889455,"scheduling_latency_s":117.889546} gitlab # [ 221.879712] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:36.800Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"1b05a32811a3ce57d27f034f","created_at":"2026-10-04T19:05:38.563Z","trace_propagation_headers":{"sentry-trace":"07b54df8b35141258dc792dc046ae114-a338b19e565d4cb6","baggage":"sentry-trace_id=07b54df8b35141258dc792dc046ae114,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"972d644ddec575a5ea7fa645b2e6543f","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T19:05:38.574Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-1b05a32811a3ce57d27f034f: done: 0.33718 sec","job_status":"done","queue_duration_s":117.889455,"scheduling_latency_s":117.889546,"redis_calls":31,"redis_duration_s":0.016658,"redis_read_bytes":444,"redis_write_bytes":3725,"redis_feature_flag_calls":20,"redis_feature_flag_duration_s":0.007816,"redis_feature_flag_read_bytes":434,"redis_feature_flag_write_bytes":2478,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002904,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005938,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.185792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.1671359,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.33718,"completed_at":"2026-10-04T19:07:36.800Z","load_balancing_strategy":"primary","db_duration_s":0.076953,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 224.923216] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.844Z","pid":1855,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 224.923451] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.844Z","pid":1855,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 224.923549] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.844Z","pid":1855,"message":"! #\u003cThread:0x0000f61b3cc71ea8 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 224.923613] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.844Z","pid":1855,"message":"! #\u003cThread:0x0000f61b3ce619c0 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 224.923711] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.845Z","pid":1855,"message":"Use Ctrl-C to stop"} gitlab # [ 224.923783] bundle[1855]: {"timestamp":"2026-10-04T19:07:39.845Z","pid":1855,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 225.608506] bundle[1855]: {"timestamp":"2026-10-04T19:07:40.528Z","pid":1855,"message":"- Worker 0 (PID: 2959) booted in 0.64s, phase: 0"} gitlab # [ 225.661666] bundle[1855]: {"timestamp":"2026-10-04T19:07:40.583Z","pid":1855,"message":"- Worker 1 (PID: 2963) booted in 0.66s, phase: 0"} gitlab # [ 225.666535] systemd[1]: Started gitlab.service. gitlab # [ 225.667820] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 225.670785] systemd[1]: Starting setup-gitlab.service... gitlab # [ 225.713713] setup-gitlab-start[2987]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 230.496732] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:07:45 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 4705 gitlab # [ 230.500708] setup-gitlab-start[2987]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 230.502003] setup-gitlab-start[2987]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 230.512853] runuser[3022]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 231.081014] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.084538] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.087774] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.971450] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.971830] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 231.978670] setup-gitlab-start[3023]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 233.975271] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:48.896Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"170a3191c39ddb8f54670303","created_at":"2026-10-04T19:07:48.827Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9f1f426a21dbee99986d3c35aabf57f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.894Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-170a3191c39ddb8f54670303: start","job_status":"start","queue_duration_s":0.002231,"scheduling_latency_s":0.002273} gitlab # [ 234.085308] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"170a3191c39ddb8f54670303","created_at":"2026-10-04T19:07:48.827Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9f1f426a21dbee99986d3c35aabf57f","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.894Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-170a3191c39ddb8f54670303: done: 0.110019 sec","job_status":"done","queue_duration_s":0.002231,"scheduling_latency_s":0.002273,"redis_calls":12,"redis_duration_s":0.009892,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001141,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004594,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004157,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00132054,"exclusive_lock_hold_duration_s":0.01328582,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.110019,"completed_at":"2026-10-04T19:07:49.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.029947,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.154990] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.014Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dbb17ea99f8fd617834d28bd","created_at":"2026-10-04T19:07:48.901Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6263c5916caaf1b87f447b76fdaa0605","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:07:48.910Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dbb17ea99f8fd617834d28bd: start","job_status":"start","queue_duration_s":0.103601,"scheduling_latency_s":0.103633} gitlab # [ 234.174308] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.056Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 234.179494] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"dbb17ea99f8fd617834d28bd","created_at":"2026-10-04T19:07:48.901Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6263c5916caaf1b87f447b76fdaa0605","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:07:48.910Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-dbb17ea99f8fd617834d28bd: done: 0.058144 sec","job_status":"done","queue_duration_s":0.103601,"scheduling_latency_s":0.103633,"redis_calls":10,"redis_duration_s":0.022962999999999997,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.013123,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005428,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019511,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036512,"duration_s":0.058144,"completed_at":"2026-10-04T19:07:49.072Z","load_balancing_strategy":"primary","db_duration_s":0.00172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.227121] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"989cfbe7304f4095e13a78ae","created_at":"2026-10-04T19:07:48.933Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38e46dfb97abbaf7f031e6fda39c7685","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:07:48.940Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-989cfbe7304f4095e13a78ae: start","job_status":"start","queue_duration_s":0.133211,"scheduling_latency_s":0.133236} gitlab # [ 234.292480] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"989cfbe7304f4095e13a78ae","created_at":"2026-10-04T19:07:48.933Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38e46dfb97abbaf7f031e6fda39c7685","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:07:48.940Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-989cfbe7304f4095e13a78ae: done: 0.13994 sec","job_status":"done","queue_duration_s":0.133211,"scheduling_latency_s":0.133236,"redis_calls":10,"redis_duration_s":0.012314,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000658,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.005899,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005757,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00413654,"duration_s":0.13994,"completed_at":"2026-10-04T19:07:49.213Z","load_balancing_strategy":"primary","db_duration_s":0.046595,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.352510] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.226Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3195b2ad7a4b7f7aeb8c3f4","created_at":"2026-10-04T19:07:48.952Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66d3d7e15743c86519d9a3d101e489e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.960Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3195b2ad7a4b7f7aeb8c3f4: start","job_status":"start","queue_duration_s":0.266134,"scheduling_latency_s":0.26617} gitlab # [ 234.535441] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.456Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"f3195b2ad7a4b7f7aeb8c3f4","created_at":"2026-10-04T19:07:48.952Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"66d3d7e15743c86519d9a3d101e489e9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.960Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-f3195b2ad7a4b7f7aeb8c3f4: done: 0.229998 sec","job_status":"done","queue_duration_s":0.266134,"scheduling_latency_s":0.26617,"redis_calls":14,"redis_duration_s":0.018276,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.002774,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011271,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004231,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.087947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00104138,"exclusive_lock_hold_duration_s":0.11858478,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.229998,"completed_at":"2026-10-04T19:07:49.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.037289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.607346] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"63d3f38a17fa9099efeb2794","created_at":"2026-10-04T19:07:48.977Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6be7f71d0049422b8900975e07f8a4db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.988Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-63d3f38a17fa9099efeb2794: start","job_status":"start","queue_duration_s":0.470094,"scheduling_latency_s":0.470129} gitlab # [ 234.805169] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.726Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"63d3f38a17fa9099efeb2794","created_at":"2026-10-04T19:07:48.977Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6be7f71d0049422b8900975e07f8a4db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:48.988Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-63d3f38a17fa9099efeb2794: done: 0.267942 sec","job_status":"done","queue_duration_s":0.470094,"scheduling_latency_s":0.470129,"redis_calls":12,"redis_duration_s":0.002628,"redis_read_bytes":11,"redis_write_bytes":1370,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00042,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001193,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001015,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.033,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.218791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007465,"exclusive_lock_hold_duration_s":0.2387974,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.267942,"completed_at":"2026-10-04T19:07:49.726Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.874057] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"106b0aaab4392e2feec467a5","created_at":"2026-10-04T19:07:49.002Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.013Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-106b0aaab4392e2feec467a5: start","job_status":"start","queue_duration_s":0.714917,"scheduling_latency_s":0.71494} gitlab # [ 234.890439] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"106b0aaab4392e2feec467a5","created_at":"2026-10-04T19:07:49.002Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.013Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-106b0aaab4392e2feec467a5: done: 0.053622 sec","job_status":"done","queue_duration_s":0.714917,"scheduling_latency_s":0.71494,"redis_calls":19,"redis_duration_s":0.004542,"redis_read_bytes":22,"redis_write_bytes":4785,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000875,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000527,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002345,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000795,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039614,"duration_s":0.053622,"completed_at":"2026-10-04T19:07:49.781Z","load_balancing_strategy":"primary","db_duration_s":0.00265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.935667] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:49.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"8b40e915231d79b2c2312d23","created_at":"2026-10-04T19:07:49.029Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"845d8416366d8b32aa0d67e0fcd1a716","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.044Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-8b40e915231d79b2c2312d23: start","job_status":"start","queue_duration_s":0.738919,"scheduling_latency_s":0.738944} gitlab # [ 235.085285] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.006Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"8b40e915231d79b2c2312d23","created_at":"2026-10-04T19:07:49.029Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"845d8416366d8b32aa0d67e0fcd1a716","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.044Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-8b40e915231d79b2c2312d23: done: 0.222408 sec","job_status":"done","queue_duration_s":0.738919,"scheduling_latency_s":0.738944,"redis_calls":12,"redis_duration_s":0.003412,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001079,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193409,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00064432,"exclusive_lock_hold_duration_s":0.19516382,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.222408,"completed_at":"2026-10-04T19:07:50.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.020282,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null} gitlab # [ 235.153971] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"90d9fcd6ff59bbd7a37afc7f","created_at":"2026-10-04T19:07:49.054Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e7da6ddfc0cb2cd701fd7d8ce259e31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.076Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-90d9fcd6ff59bbd7a37afc7f: start","job_status":"start","queue_duration_s":0.931452,"scheduling_latency_s":0.931478} gitlab # [ 235.176071] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.023Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 235.180776] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"90d9fcd6ff59bbd7a37afc7f","created_at":"2026-10-04T19:07:49.054Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8e7da6ddfc0cb2cd701fd7d8ce259e31","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.076Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-90d9fcd6ff59bbd7a37afc7f: done: 0.023856 sec","job_status":"done","queue_duration_s":0.931452,"scheduling_latency_s":0.931478,"redis_calls":10,"redis_duration_s":0.002062,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000489,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000393,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013585,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030722,"duration_s":0.023856,"completed_at":"2026-10-04T19:07:50.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000847,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.229586] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.033Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7b860c4f7ff0ecb726720f60","created_at":"2026-10-04T19:07:49.099Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ac2f3aabcd92e1b9fae3e245ec4a786","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.123Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7b860c4f7ff0ecb726720f60: start","job_status":"start","queue_duration_s":0.910649,"scheduling_latency_s":0.91067} gitlab # [ 235.244090] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.061Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7b860c4f7ff0ecb726720f60","created_at":"2026-10-04T19:07:49.099Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ac2f3aabcd92e1b9fae3e245ec4a786","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.123Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7b860c4f7ff0ecb726720f60: done: 0.028016 sec","job_status":"done","queue_duration_s":0.910649,"scheduling_latency_s":0.91067,"redis_calls":10,"redis_duration_s":0.0019219999999999999,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000385,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016698,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028538,"duration_s":0.028016,"completed_at":"2026-10-04T19:07:50.061Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003721,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.287610] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e0a41c5e4206c6b1eb6d2450","created_at":"2026-10-04T19:07:49.125Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"683898d0bfc99b780af3a6cb003e7718","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.132Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e0a41c5e4206c6b1eb6d2450: start","job_status":"start","queue_duration_s":0.930929,"scheduling_latency_s":0.930946} gitlab # [ 235.302678] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.099Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"e0a41c5e4206c6b1eb6d2450","created_at":"2026-10-04T19:07:49.125Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"683898d0bfc99b780af3a6cb003e7718","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.132Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-e0a41c5e4206c6b1eb6d2450: done: 0.036709 sec","job_status":"done","queue_duration_s":0.930929,"scheduling_latency_s":0.930946,"redis_calls":13,"redis_duration_s":0.003441,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00212,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.010934540000000001,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.036709,"completed_at":"2026-10-04T19:07:50.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003774,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 235.348572] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.101Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"33107c4c4752c955dd1d95ab","created_at":"2026-10-04T19:07:49.153Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"182cb6c8ec9305295ce826bc4b0e4913","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.164Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-33107c4c4752c955dd1d95ab: start","job_status":"start","queue_duration_s":0.93716,"scheduling_latency_s":0.937179} gitlab # [ 235.363157] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.148Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"33107c4c4752c955dd1d95ab","created_at":"2026-10-04T19:07:49.153Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"182cb6c8ec9305295ce826bc4b0e4913","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.164Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-33107c4c4752c955dd1d95ab: done: 0.046961 sec","job_status":"done","queue_duration_s":0.93716,"scheduling_latency_s":0.937179,"redis_calls":16,"redis_duration_s":0.005869,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001528,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000876,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003465,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025181,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002518,"duration_s":0.046961,"completed_at":"2026-10-04T19:07:50.148Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.407790] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.149Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d5b72e39cf38049b0aea7d01","created_at":"2026-10-04T19:07:49.167Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.176Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d5b72e39cf38049b0aea7d01: start","job_status":"start","queue_duration_s":0.973331,"scheduling_latency_s":0.973351} gitlab # [ 235.421876] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d5b72e39cf38049b0aea7d01","created_at":"2026-10-04T19:07:49.167Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.176Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d5b72e39cf38049b0aea7d01: done: 0.114209 sec","job_status":"done","queue_duration_s":0.973331,"scheduling_latency_s":0.973351,"redis_calls":71,"redis_duration_s":0.01124,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001498,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.00085,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.008429,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029612,"duration_s":0.114209,"completed_at":"2026-10-04T19:07:50.263Z","load_balancing_strategy":"primary","db_duration_s":0.001774,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.466057] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a4e0033119831352c14a1be6","created_at":"2026-10-04T19:07:49.180Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12ca68d59f3dacc52b701bcf152b9bfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.193Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a4e0033119831352c14a1be6: start","job_status":"start","queue_duration_s":1.07186,"scheduling_latency_s":1.071879} gitlab # [ 235.480913] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.288Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"a4e0033119831352c14a1be6","created_at":"2026-10-04T19:07:49.180Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12ca68d59f3dacc52b701bcf152b9bfd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.193Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-a4e0033119831352c14a1be6: done: 0.023795 sec","job_status":"done","queue_duration_s":1.07186,"scheduling_latency_s":1.071879,"redis_calls":12,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":11,"redis_write_bytes":1482,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000618,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000809,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028014,"duration_s":0.023795,"completed_at":"2026-10-04T19:07:50.288Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000714,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.524801] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b0eb9ef031fd3fe943bc42ce","created_at":"2026-10-04T19:07:49.208Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23c222154592a1dac0919509cc5d6511","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.217Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b0eb9ef031fd3fe943bc42ce: start","job_status":"start","queue_duration_s":1.073156,"scheduling_latency_s":1.073176} gitlab # [ 235.539869] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.310Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b0eb9ef031fd3fe943bc42ce","created_at":"2026-10-04T19:07:49.208Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23c222154592a1dac0919509cc5d6511","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.217Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b0eb9ef031fd3fe943bc42ce: done: 0.020263 sec","job_status":"done","queue_duration_s":1.073156,"scheduling_latency_s":1.073176,"redis_calls":10,"redis_duration_s":0.001395,"redis_read_bytes":9,"redis_write_bytes":1299,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000296,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000297,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024726,"duration_s":0.020263,"completed_at":"2026-10-04T19:07:50.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000313,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.583936] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.312Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"055b6191684770118d1e5199","created_at":"2026-10-04T19:07:49.225Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bd0ab766a5f46b128e16e05fb4dc902","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.233Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-055b6191684770118d1e5199: start","job_status":"start","queue_duration_s":1.078376,"scheduling_latency_s":1.078393} gitlab # [ 235.600190] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.357Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"055b6191684770118d1e5199","created_at":"2026-10-04T19:07:49.225Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7bd0ab766a5f46b128e16e05fb4dc902","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.233Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-055b6191684770118d1e5199: done: 0.044928 sec","job_status":"done","queue_duration_s":1.078376,"scheduling_latency_s":1.078393,"redis_calls":12,"redis_duration_s":0.002363,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000302,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063658,"exclusive_lock_hold_duration_s":0.016414179999999997,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.044928,"completed_at":"2026-10-04T19:07:50.357Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012936,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.646363] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.359Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"252ef99ede628a1f5d6e8c7c","created_at":"2026-10-04T19:07:49.260Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8633b19c1b1d7064b17299540291a23f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.265Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-252ef99ede628a1f5d6e8c7c: start","job_status":"start","queue_duration_s":1.093137,"scheduling_latency_s":1.093162} gitlab # [ 235.660553] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"252ef99ede628a1f5d6e8c7c","created_at":"2026-10-04T19:07:49.260Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8633b19c1b1d7064b17299540291a23f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.265Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-252ef99ede628a1f5d6e8c7c: done: 0.0356 sec","job_status":"done","queue_duration_s":1.093137,"scheduling_latency_s":1.093162,"redis_calls":12,"redis_duration_s":0.002533,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000445,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00071754,"exclusive_lock_hold_duration_s":0.00909184,"duration_s":0.0356,"completed_at":"2026-10-04T19:07:50.394Z","load_balancing_strategy":"primary","db_duration_s":0.003686,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.703050] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.396Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b3a58a3677da6264483e1353","created_at":"2026-10-04T19:07:49.269Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4551543fcfde0cb647b4fdc32412902","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.282Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b3a58a3677da6264483e1353: start","job_status":"start","queue_duration_s":1.113534,"scheduling_latency_s":1.113557} gitlab # [ 235.717524] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.444Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b3a58a3677da6264483e1353","created_at":"2026-10-04T19:07:49.269Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a4551543fcfde0cb647b4fdc32412902","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.282Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b3a58a3677da6264483e1353: done: 0.048318 sec","job_status":"done","queue_duration_s":1.113534,"scheduling_latency_s":1.113557,"redis_calls":10,"redis_duration_s":0.002332,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001417,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000493,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041422,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.048318,"completed_at":"2026-10-04T19:07:50.444Z","load_balancing_strategy":"primary","db_duration_s":0.016757,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.761167] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"813844bab867c62bc59121b6","created_at":"2026-10-04T19:07:49.287Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1be1f7ccabdb89b5a67f3ccb409e8e8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.300Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-813844bab867c62bc59121b6: start","job_status":"start","queue_duration_s":1.146331,"scheduling_latency_s":1.146358} gitlab # [ 235.979703] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"813844bab867c62bc59121b6","created_at":"2026-10-04T19:07:49.287Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1be1f7ccabdb89b5a67f3ccb409e8e8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.300Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-813844bab867c62bc59121b6: done: 0.45426 sec","job_status":"done","queue_duration_s":1.146331,"scheduling_latency_s":1.146358,"redis_calls":565,"redis_duration_s":0.069067,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000442,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001743,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.066882,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.418179,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003019,"duration_s":0.45426,"completed_at":"2026-10-04T19:07:50.901Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000518,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.046914] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.902Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d645d2b51cda65b16226b54d","created_at":"2026-10-04T19:07:49.324Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e5b10550c39f140956916d5cab3a82e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.359Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d645d2b51cda65b16226b54d: start","job_status":"start","queue_duration_s":1.543257,"scheduling_latency_s":1.543277} gitlab # [ 236.065752] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.915Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 236.069713] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d645d2b51cda65b16226b54d","created_at":"2026-10-04T19:07:49.324Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e5b10550c39f140956916d5cab3a82e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.359Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d645d2b51cda65b16226b54d: done: 0.02192 sec","job_status":"done","queue_duration_s":1.543257,"scheduling_latency_s":1.543277,"redis_calls":10,"redis_duration_s":0.0018599999999999999,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000355,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012032,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002833,"duration_s":0.02192,"completed_at":"2026-10-04T19:07:50.924Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.115003] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.927Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"491a8818f3b8fddfa4f14dab","created_at":"2026-10-04T19:07:49.368Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00a19e85eb5bd9e4dc03bdf34ea1b936","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.379Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-491a8818f3b8fddfa4f14dab: start","job_status":"start","queue_duration_s":1.548217,"scheduling_latency_s":1.548244} gitlab # [ 236.130303] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.947Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 236.134232] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"491a8818f3b8fddfa4f14dab","created_at":"2026-10-04T19:07:49.368Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00a19e85eb5bd9e4dc03bdf34ea1b936","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.379Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-491a8818f3b8fddfa4f14dab: done: 0.029436 sec","job_status":"done","queue_duration_s":1.548217,"scheduling_latency_s":1.548244,"redis_calls":10,"redis_duration_s":0.0021709999999999998,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000498,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001202,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017676,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035224,"duration_s":0.029436,"completed_at":"2026-10-04T19:07:50.956Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002045,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.178172] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.959Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"01f298079a1782851a3ffbcc","created_at":"2026-10-04T19:07:49.392Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba43cf668e3339aa463e8b91153be48","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.400Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-01f298079a1782851a3ffbcc: start","job_status":"start","queue_duration_s":1.559367,"scheduling_latency_s":1.559392} gitlab # [ 236.193222] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.975Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 236.196862] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.985Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"01f298079a1782851a3ffbcc","created_at":"2026-10-04T19:07:49.392Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cba43cf668e3339aa463e8b91153be48","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.400Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-01f298079a1782851a3ffbcc: done: 0.025971 sec","job_status":"done","queue_duration_s":1.559367,"scheduling_latency_s":1.559392,"redis_calls":10,"redis_duration_s":0.001881,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001101,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0004,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033044,"duration_s":0.025971,"completed_at":"2026-10-04T19:07:50.985Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000476,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.241597] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:50.987Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e6c7e1dc626d03d9f58d3467","created_at":"2026-10-04T19:07:49.405Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b583537508195f78fdff8f626adad921","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:07:49.419Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e6c7e1dc626d03d9f58d3467: start","job_status":"start","queue_duration_s":1.567696,"scheduling_latency_s":1.567716} gitlab # [ 236.255573] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.000Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 236.259192] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"e6c7e1dc626d03d9f58d3467","created_at":"2026-10-04T19:07:49.405Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b583537508195f78fdff8f626adad921","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:07:49.419Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-e6c7e1dc626d03d9f58d3467: done: 0.021328 sec","job_status":"done","queue_duration_s":1.567696,"scheduling_latency_s":1.567716,"redis_calls":10,"redis_duration_s":0.0020039999999999997,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001192,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032572,"duration_s":0.021328,"completed_at":"2026-10-04T19:07:51.008Z","load_balancing_strategy":"primary","db_duration_s":0.000407,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.302179] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.009Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3a5b3099db87b6e4e1320b74","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3a5b3099db87b6e4e1320b74: start","job_status":"start","queue_duration_s":1.239233,"scheduling_latency_s":1.23925} gitlab # [ 236.315043] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.027Z","message":"No existing merge request to be cleaned up."} gitlab # [ 236.316807] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.045Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"3a5b3099db87b6e4e1320b74","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-3a5b3099db87b6e4e1320b74: done: 0.036284 sec","job_status":"done","queue_duration_s":1.239233,"scheduling_latency_s":1.23925,"redis_calls":13,"redis_duration_s":0.00297,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000342,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001131,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001497,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041326,"duration_s":0.036284,"completed_at":"2026-10-04T19:07:51.045Z","load_balancing_strategy":"primary","db_duration_s":0.003868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.358054] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.047Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5d65d2ece2d1932a5abb8159","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5d65d2ece2d1932a5abb8159: start","job_status":"start","queue_duration_s":1.277185,"scheduling_latency_s":1.277209} gitlab # [ 236.370920] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.062Z","message":"No existing merge request to be cleaned up."} gitlab # [ 236.372717] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.074Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5d65d2ece2d1932a5abb8159","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5d65d2ece2d1932a5abb8159: done: 0.026496 sec","job_status":"done","queue_duration_s":1.277185,"scheduling_latency_s":1.277209,"redis_calls":10,"redis_duration_s":0.002251,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026496,"completed_at":"2026-10-04T19:07:51.073Z","load_balancing_strategy":"primary","db_duration_s":0.001988,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.413431] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.075Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4a8cb341860a0f11ac01be0c","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4a8cb341860a0f11ac01be0c: start","job_status":"start","queue_duration_s":1.305359,"scheduling_latency_s":1.305384} gitlab # [ 236.426147] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.090Z","message":"No existing merge request to be cleaned up."} gitlab # [ 236.427839] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.099Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4a8cb341860a0f11ac01be0c","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4a8cb341860a0f11ac01be0c: done: 0.024215 sec","job_status":"done","queue_duration_s":1.305359,"scheduling_latency_s":1.305384,"redis_calls":10,"redis_duration_s":0.002029,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000858,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001171,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024215,"completed_at":"2026-10-04T19:07:51.099Z","load_balancing_strategy":"primary","db_duration_s":0.001608,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.468041] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.101Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"51884cfbccf228c20aa53305","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-51884cfbccf228c20aa53305: start","job_status":"start","queue_duration_s":1.331157,"scheduling_latency_s":1.331178} gitlab # [ 236.480744] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.114Z","message":"No existing merge request to be cleaned up."} gitlab # [ 236.482463] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.123Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"51884cfbccf228c20aa53305","created_at":"2026-10-04T19:07:49.750Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-bf882a92ab2447f0","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bf8b5800449ea17d57d8607fada114f1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:49.770Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-51884cfbccf228c20aa53305: done: 0.021958 sec","job_status":"done","queue_duration_s":1.331157,"scheduling_latency_s":1.331178,"redis_calls":10,"redis_duration_s":0.001868,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000786,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001082,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014658,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021958,"completed_at":"2026-10-04T19:07:51.123Z","load_balancing_strategy":"primary","db_duration_s":0.001554,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.522336] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ff6bdd8087cd80dc23f8726","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ff6bdd8087cd80dc23f8726: start","job_status":"start","queue_duration_s":0.870375,"scheduling_latency_s":0.870393} gitlab # [ 236.535351] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.147Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ff6bdd8087cd80dc23f8726","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ff6bdd8087cd80dc23f8726: done: 0.023005 sec","job_status":"done","queue_duration_s":0.870375,"scheduling_latency_s":0.870393,"redis_calls":16,"redis_duration_s":0.0031940000000000002,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000911,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000778,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001505,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026232,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023005,"completed_at":"2026-10-04T19:07:51.147Z","load_balancing_strategy":"primary","db_duration_s":0.000438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.578343] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7994888b248f39b2b82de5","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7994888b248f39b2b82de5: start","job_status":"start","queue_duration_s":0.89453,"scheduling_latency_s":0.894546} gitlab # [ 236.591293] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4d7994888b248f39b2b82de5","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4d7994888b248f39b2b82de5: done: 0.016359 sec","job_status":"done","queue_duration_s":0.89453,"scheduling_latency_s":0.894546,"redis_calls":13,"redis_duration_s":0.0023,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001365,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016359,"completed_at":"2026-10-04T19:07:51.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.633287] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69936e63c81dd2ca2d2a98e","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69936e63c81dd2ca2d2a98e: start","job_status":"start","queue_duration_s":0.916822,"scheduling_latency_s":0.916842} gitlab # [ 236.646031] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.191Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69936e63c81dd2ca2d2a98e","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69936e63c81dd2ca2d2a98e: done: 0.020728 sec","job_status":"done","queue_duration_s":0.916822,"scheduling_latency_s":0.916842,"redis_calls":13,"redis_duration_s":0.005504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001715,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003789,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020728,"completed_at":"2026-10-04T19:07:51.191Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.686619] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.193Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11df0f84b5b4a55a44d1cb99","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11df0f84b5b4a55a44d1cb99: start","job_status":"start","queue_duration_s":0.93882,"scheduling_latency_s":0.938839} gitlab # [ 236.699548] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"11df0f84b5b4a55a44d1cb99","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-11df0f84b5b4a55a44d1cb99: done: 0.021884 sec","job_status":"done","queue_duration_s":0.93882,"scheduling_latency_s":0.938839,"redis_calls":13,"redis_duration_s":0.0055569999999999994,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.004193,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001364,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01061,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021884,"completed_at":"2026-10-04T19:07:51.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.740340] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78bde18ebdaea4cdd5031c9d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78bde18ebdaea4cdd5031c9d: start","job_status":"start","queue_duration_s":0.962925,"scheduling_latency_s":0.962943} gitlab # [ 236.753169] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78bde18ebdaea4cdd5031c9d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78bde18ebdaea4cdd5031c9d: done: 0.015877 sec","job_status":"done","queue_duration_s":0.962925,"scheduling_latency_s":0.962943,"redis_calls":13,"redis_duration_s":0.002006,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001288,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015877,"completed_at":"2026-10-04T19:07:51.233Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.793581] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.234Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db37d154041fb0d4580fbc05","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db37d154041fb0d4580fbc05: start","job_status":"start","queue_duration_s":0.980066,"scheduling_latency_s":0.980084} gitlab # [ 236.806485] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"db37d154041fb0d4580fbc05","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-db37d154041fb0d4580fbc05: done: 0.015591 sec","job_status":"done","queue_duration_s":0.980066,"scheduling_latency_s":0.980084,"redis_calls":13,"redis_duration_s":0.0019429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001282,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015591,"completed_at":"2026-10-04T19:07:51.250Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.848484] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"004c43a3e96c2bb6945594bd","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-004c43a3e96c2bb6945594bd: start","job_status":"start","queue_duration_s":0.996839,"scheduling_latency_s":0.996855} gitlab # [ 236.861065] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"004c43a3e96c2bb6945594bd","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-004c43a3e96c2bb6945594bd: done: 0.016152 sec","job_status":"done","queue_duration_s":0.996839,"scheduling_latency_s":0.996855,"redis_calls":13,"redis_duration_s":0.001902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001234,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016152,"completed_at":"2026-10-04T19:07:51.267Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.901600] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.268Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbc9e8042900dac69d0846b6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbc9e8042900dac69d0846b6: start","job_status":"start","queue_duration_s":1.014144,"scheduling_latency_s":1.01416} gitlab # [ 236.914464] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbc9e8042900dac69d0846b6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbc9e8042900dac69d0846b6: done: 0.014658 sec","job_status":"done","queue_duration_s":1.014144,"scheduling_latency_s":1.01416,"redis_calls":13,"redis_duration_s":0.00203,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014658,"completed_at":"2026-10-04T19:07:51.283Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.955020] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d816e9d85029767c7d054ec6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d816e9d85029767c7d054ec6: start","job_status":"start","queue_duration_s":1.030766,"scheduling_latency_s":1.030782} gitlab # [ 236.968097] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d816e9d85029767c7d054ec6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d816e9d85029767c7d054ec6: done: 0.015799 sec","job_status":"done","queue_duration_s":1.030766,"scheduling_latency_s":1.030782,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010731,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015799,"completed_at":"2026-10-04T19:07:51.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.008736] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f5d43fbd3c5390cd712952a","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f5d43fbd3c5390cd712952a: start","job_status":"start","queue_duration_s":1.047717,"scheduling_latency_s":1.047734} gitlab # [ 237.022549] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f5d43fbd3c5390cd712952a","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f5d43fbd3c5390cd712952a: done: 0.015588 sec","job_status":"done","queue_duration_s":1.047717,"scheduling_latency_s":1.047734,"redis_calls":13,"redis_duration_s":0.0019299999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000667,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001263,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010827,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015588,"completed_at":"2026-10-04T19:07:51.317Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.063274] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5b4054f0e080de705cc46d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5b4054f0e080de705cc46d: start","job_status":"start","queue_duration_s":1.064486,"scheduling_latency_s":1.064502} gitlab # [ 237.076288] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b5b4054f0e080de705cc46d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b5b4054f0e080de705cc46d: done: 0.01546 sec","job_status":"done","queue_duration_s":1.064486,"scheduling_latency_s":1.064502,"redis_calls":13,"redis_duration_s":0.002088,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009968,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01546,"completed_at":"2026-10-04T19:07:51.334Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.116968] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.336Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3b749aef7e8843b851392e9","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3b749aef7e8843b851392e9: start","job_status":"start","queue_duration_s":1.081874,"scheduling_latency_s":1.081891} gitlab # [ 237.129768] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3b749aef7e8843b851392e9","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3b749aef7e8843b851392e9: done: 0.015814 sec","job_status":"done","queue_duration_s":1.081874,"scheduling_latency_s":1.081891,"redis_calls":13,"redis_duration_s":0.0020610000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001257,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010848,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015814,"completed_at":"2026-10-04T19:07:51.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.170541] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6539b2b40d23f68a74f54e3c","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6539b2b40d23f68a74f54e3c: start","job_status":"start","queue_duration_s":1.098894,"scheduling_latency_s":1.098911} gitlab # [ 237.183275] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6539b2b40d23f68a74f54e3c","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6539b2b40d23f68a74f54e3c: done: 0.015984 sec","job_status":"done","queue_duration_s":1.098894,"scheduling_latency_s":1.098911,"redis_calls":13,"redis_duration_s":0.0019590000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011067,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015984,"completed_at":"2026-10-04T19:07:51.369Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.224052] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06b37a2d2d9b6733be3e8f98","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06b37a2d2d9b6733be3e8f98: start","job_status":"start","queue_duration_s":1.116002,"scheduling_latency_s":1.116019} gitlab # [ 237.237519] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06b37a2d2d9b6733be3e8f98","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06b37a2d2d9b6733be3e8f98: done: 0.015366 sec","job_status":"done","queue_duration_s":1.116002,"scheduling_latency_s":1.116019,"redis_calls":13,"redis_duration_s":0.001864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001225,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015366,"completed_at":"2026-10-04T19:07:51.385Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.277831] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6efd06e733999ad6e93e8de2","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6efd06e733999ad6e93e8de2: start","job_status":"start","queue_duration_s":1.133177,"scheduling_latency_s":1.133194} gitlab # [ 237.290735] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.402Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6efd06e733999ad6e93e8de2","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6efd06e733999ad6e93e8de2: done: 0.014957 sec","job_status":"done","queue_duration_s":1.133177,"scheduling_latency_s":1.133194,"redis_calls":13,"redis_duration_s":0.0019069999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000666,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014957,"completed_at":"2026-10-04T19:07:51.402Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.331667] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318dcf77e6bd19aed6bb072d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318dcf77e6bd19aed6bb072d: start","job_status":"start","queue_duration_s":1.149354,"scheduling_latency_s":1.14937} gitlab # [ 237.344646] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"318dcf77e6bd19aed6bb072d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-318dcf77e6bd19aed6bb072d: done: 0.015025 sec","job_status":"done","queue_duration_s":1.149354,"scheduling_latency_s":1.14937,"redis_calls":13,"redis_duration_s":0.0020670000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015025,"completed_at":"2026-10-04T19:07:51.418Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.385069] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e17a82657dcb2edbc8decf34","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e17a82657dcb2edbc8decf34: start","job_status":"start","queue_duration_s":1.165555,"scheduling_latency_s":1.165571} gitlab # [ 237.397906] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e17a82657dcb2edbc8decf34","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e17a82657dcb2edbc8decf34: done: 0.0145 sec","job_status":"done","queue_duration_s":1.165555,"scheduling_latency_s":1.165571,"redis_calls":13,"redis_duration_s":0.001864,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00122,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0145,"completed_at":"2026-10-04T19:07:51.434Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.439199] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b86118de4482f1f01669cc9","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b86118de4482f1f01669cc9: start","job_status":"start","queue_duration_s":1.181224,"scheduling_latency_s":1.181242} gitlab # [ 237.452173] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b86118de4482f1f01669cc9","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b86118de4482f1f01669cc9: done: 0.01454 sec","job_status":"done","queue_duration_s":1.181224,"scheduling_latency_s":1.181242,"redis_calls":13,"redis_duration_s":0.00185,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01454,"completed_at":"2026-10-04T19:07:51.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.492566] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf05f02eda4d57b4e9880b62","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf05f02eda4d57b4e9880b62: start","job_status":"start","queue_duration_s":1.197207,"scheduling_latency_s":1.197224} gitlab # [ 237.505369] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf05f02eda4d57b4e9880b62","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf05f02eda4d57b4e9880b62: done: 0.014464 sec","job_status":"done","queue_duration_s":1.197207,"scheduling_latency_s":1.197224,"redis_calls":13,"redis_duration_s":0.001892,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001245,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009546,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014464,"completed_at":"2026-10-04T19:07:51.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.545984] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d2c98f48be239de5d85e3ea","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d2c98f48be239de5d85e3ea: start","job_status":"start","queue_duration_s":1.214424,"scheduling_latency_s":1.214442} gitlab # [ 237.558881] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.483Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d2c98f48be239de5d85e3ea","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d2c98f48be239de5d85e3ea: done: 0.014861 sec","job_status":"done","queue_duration_s":1.214424,"scheduling_latency_s":1.214442,"redis_calls":13,"redis_duration_s":0.00193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001239,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014861,"completed_at":"2026-10-04T19:07:51.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.599530] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7fce3f69778f306da6982e","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7fce3f69778f306da6982e: start","job_status":"start","queue_duration_s":1.230435,"scheduling_latency_s":1.230452} gitlab # [ 237.612552] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b7fce3f69778f306da6982e","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b7fce3f69778f306da6982e: done: 0.014501 sec","job_status":"done","queue_duration_s":1.230435,"scheduling_latency_s":1.230452,"redis_calls":13,"redis_duration_s":0.001751,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009786,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014501,"completed_at":"2026-10-04T19:07:51.499Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.654002] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56d9539753e24d73ee098e4d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56d9539753e24d73ee098e4d: start","job_status":"start","queue_duration_s":1.246141,"scheduling_latency_s":1.246157} gitlab # [ 237.666810] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56d9539753e24d73ee098e4d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56d9539753e24d73ee098e4d: done: 0.018891 sec","job_status":"done","queue_duration_s":1.246141,"scheduling_latency_s":1.246157,"redis_calls":13,"redis_duration_s":0.001875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001203,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009757,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018891,"completed_at":"2026-10-04T19:07:51.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.707456] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a3491a4cd1b5e658cdf292","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a3491a4cd1b5e658cdf292: start","job_status":"start","queue_duration_s":1.266327,"scheduling_latency_s":1.266345} gitlab # [ 237.720438] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a3491a4cd1b5e658cdf292","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a3491a4cd1b5e658cdf292: done: 0.014584 sec","job_status":"done","queue_duration_s":1.266327,"scheduling_latency_s":1.266345,"redis_calls":13,"redis_duration_s":0.001889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000736,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001153,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014584,"completed_at":"2026-10-04T19:07:51.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.761090] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4abb403a0d5db5f577354ad4","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4abb403a0d5db5f577354ad4: start","job_status":"start","queue_duration_s":1.282156,"scheduling_latency_s":1.282172} gitlab # [ 237.773883] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4abb403a0d5db5f577354ad4","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4abb403a0d5db5f577354ad4: done: 0.015184 sec","job_status":"done","queue_duration_s":1.282156,"scheduling_latency_s":1.282172,"redis_calls":13,"redis_duration_s":0.0023250000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001623,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015184,"completed_at":"2026-10-04T19:07:51.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.814868] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f68e13f53da3a67dde08e338","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f68e13f53da3a67dde08e338: start","job_status":"start","queue_duration_s":1.2986,"scheduling_latency_s":1.298617} gitlab # [ 237.827781] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f68e13f53da3a67dde08e338","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f68e13f53da3a67dde08e338: done: 0.014694 sec","job_status":"done","queue_duration_s":1.2986,"scheduling_latency_s":1.298617,"redis_calls":13,"redis_duration_s":0.0017720000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001136,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014694,"completed_at":"2026-10-04T19:07:51.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.869601] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26684cea32f5ea80e6a58957","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26684cea32f5ea80e6a58957: start","job_status":"start","queue_duration_s":1.314467,"scheduling_latency_s":1.314484} gitlab # [ 237.882523] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"26684cea32f5ea80e6a58957","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-26684cea32f5ea80e6a58957: done: 0.014948 sec","job_status":"done","queue_duration_s":1.314467,"scheduling_latency_s":1.314484,"redis_calls":13,"redis_duration_s":0.0018520000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001213,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014948,"completed_at":"2026-10-04T19:07:51.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.923338] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a6d13694ac3b5dba38e2a5f","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a6d13694ac3b5dba38e2a5f: start","job_status":"start","queue_duration_s":1.330863,"scheduling_latency_s":1.33088} gitlab # [ 237.936338] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a6d13694ac3b5dba38e2a5f","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a6d13694ac3b5dba38e2a5f: done: 0.014921 sec","job_status":"done","queue_duration_s":1.330863,"scheduling_latency_s":1.33088,"redis_calls":13,"redis_duration_s":0.0019709999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014921,"completed_at":"2026-10-04T19:07:51.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.976981] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"913f68f55f27f07327deda28","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-913f68f55f27f07327deda28: start","job_status":"start","queue_duration_s":1.346991,"scheduling_latency_s":1.347007} gitlab # [ 237.989830] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"913f68f55f27f07327deda28","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-913f68f55f27f07327deda28: done: 0.014999 sec","job_status":"done","queue_duration_s":1.346991,"scheduling_latency_s":1.347007,"redis_calls":13,"redis_duration_s":0.002146,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-10-04T19:07:51.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.030179] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"283212a4e3867c8bda6e5ff2","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-283212a4e3867c8bda6e5ff2: start","job_status":"start","queue_duration_s":1.363153,"scheduling_latency_s":1.36317} gitlab # [ 238.042914] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.634Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"283212a4e3867c8bda6e5ff2","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-283212a4e3867c8bda6e5ff2: done: 0.017405 sec","job_status":"done","queue_duration_s":1.363153,"scheduling_latency_s":1.36317,"redis_calls":13,"redis_duration_s":0.004332,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003651,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017405,"completed_at":"2026-10-04T19:07:51.634Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.084591] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.636Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137b121ceb9ceedb3264600d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137b121ceb9ceedb3264600d: start","job_status":"start","queue_duration_s":1.381894,"scheduling_latency_s":1.381911} gitlab # [ 238.097403] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"137b121ceb9ceedb3264600d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-137b121ceb9ceedb3264600d: done: 0.014725 sec","job_status":"done","queue_duration_s":1.381894,"scheduling_latency_s":1.381911,"redis_calls":13,"redis_duration_s":0.00184,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000645,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001195,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0096,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014725,"completed_at":"2026-10-04T19:07:51.650Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.138361] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.652Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17384617d92bf788711aee7b","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17384617d92bf788711aee7b: start","job_status":"start","queue_duration_s":1.397684,"scheduling_latency_s":1.397701} gitlab # [ 238.151216] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"17384617d92bf788711aee7b","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-17384617d92bf788711aee7b: done: 0.014618 sec","job_status":"done","queue_duration_s":1.397684,"scheduling_latency_s":1.397701,"redis_calls":13,"redis_duration_s":0.001836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00065,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009881,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014618,"completed_at":"2026-10-04T19:07:51.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.192609] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.667Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00ad353d33fb1e4a04f3c1e6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00ad353d33fb1e4a04f3c1e6: start","job_status":"start","queue_duration_s":1.413486,"scheduling_latency_s":1.413502} gitlab # [ 238.205599] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"00ad353d33fb1e4a04f3c1e6","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-00ad353d33fb1e4a04f3c1e6: done: 0.015055 sec","job_status":"done","queue_duration_s":1.413486,"scheduling_latency_s":1.413502,"redis_calls":13,"redis_duration_s":0.002101,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000722,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001379,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010112,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015055,"completed_at":"2026-10-04T19:07:51.682Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.246388] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.684Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e79fb14418877c3dbf4895","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e79fb14418877c3dbf4895: start","job_status":"start","queue_duration_s":1.429727,"scheduling_latency_s":1.429744} gitlab # [ 238.260491] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e79fb14418877c3dbf4895","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e79fb14418877c3dbf4895: done: 0.01488 sec","job_status":"done","queue_duration_s":1.429727,"scheduling_latency_s":1.429744,"redis_calls":13,"redis_duration_s":0.0019390000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01488,"completed_at":"2026-10-04T19:07:51.698Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.301099] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.700Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"925b0d4a098d63964453aef0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-925b0d4a098d63964453aef0: start","job_status":"start","queue_duration_s":1.445801,"scheduling_latency_s":1.445817} gitlab # [ 238.313941] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"925b0d4a098d63964453aef0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-925b0d4a098d63964453aef0: done: 0.015162 sec","job_status":"done","queue_duration_s":1.445801,"scheduling_latency_s":1.445817,"redis_calls":13,"redis_duration_s":0.002184,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001524,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015162,"completed_at":"2026-10-04T19:07:51.715Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.354525] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.716Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cbee8ec547376f15170d9c0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cbee8ec547376f15170d9c0: start","job_status":"start","queue_duration_s":1.462217,"scheduling_latency_s":1.462234} gitlab # [ 238.367281] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cbee8ec547376f15170d9c0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cbee8ec547376f15170d9c0: done: 0.0147 sec","job_status":"done","queue_duration_s":1.462217,"scheduling_latency_s":1.462234,"redis_calls":13,"redis_duration_s":0.001852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000653,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0147,"completed_at":"2026-10-04T19:07:51.731Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.408108] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.732Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e873a11b20c6d2acf80e1d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e873a11b20c6d2acf80e1d: start","job_status":"start","queue_duration_s":1.478185,"scheduling_latency_s":1.478202} gitlab # [ 238.420712] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e873a11b20c6d2acf80e1d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e873a11b20c6d2acf80e1d: done: 0.015479 sec","job_status":"done","queue_duration_s":1.478185,"scheduling_latency_s":1.478202,"redis_calls":13,"redis_duration_s":0.001928,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015479,"completed_at":"2026-10-04T19:07:51.748Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.461803] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb96dcfa2f6ff2680e0bb3bc","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb96dcfa2f6ff2680e0bb3bc: start","job_status":"start","queue_duration_s":1.494862,"scheduling_latency_s":1.494879} gitlab # [ 238.474498] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.765Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fb96dcfa2f6ff2680e0bb3bc","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fb96dcfa2f6ff2680e0bb3bc: done: 0.016255 sec","job_status":"done","queue_duration_s":1.494862,"scheduling_latency_s":1.494879,"redis_calls":13,"redis_duration_s":0.002084,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001307,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016255,"completed_at":"2026-10-04T19:07:51.765Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.514936] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d1d5881354829fc8f614e0d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d1d5881354829fc8f614e0d: start","job_status":"start","queue_duration_s":1.512309,"scheduling_latency_s":1.512326} gitlab # [ 238.527886] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d1d5881354829fc8f614e0d","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d1d5881354829fc8f614e0d: done: 0.016883 sec","job_status":"done","queue_duration_s":1.512309,"scheduling_latency_s":1.512326,"redis_calls":13,"redis_duration_s":0.0024010000000000004,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001721,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016883,"completed_at":"2026-10-04T19:07:51.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.568744] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3315e8ec7ea7cd3f774b055","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3315e8ec7ea7cd3f774b055: start","job_status":"start","queue_duration_s":1.530357,"scheduling_latency_s":1.530373} gitlab # [ 238.581688] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3315e8ec7ea7cd3f774b055","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3315e8ec7ea7cd3f774b055: done: 0.016138 sec","job_status":"done","queue_duration_s":1.530357,"scheduling_latency_s":1.530373,"redis_calls":13,"redis_duration_s":0.0019030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010869,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016138,"completed_at":"2026-10-04T19:07:51.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.622383] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fe5bfabda35cc7379b0bd85","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fe5bfabda35cc7379b0bd85: start","job_status":"start","queue_duration_s":1.54821,"scheduling_latency_s":1.548228} gitlab # [ 238.635166] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fe5bfabda35cc7379b0bd85","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fe5bfabda35cc7379b0bd85: done: 0.016276 sec","job_status":"done","queue_duration_s":1.54821,"scheduling_latency_s":1.548228,"redis_calls":13,"redis_duration_s":0.002147,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001223,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016276,"completed_at":"2026-10-04T19:07:51.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.676989] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.820Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7581519898dab319312721","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7581519898dab319312721: start","job_status":"start","queue_duration_s":1.566302,"scheduling_latency_s":1.566319} gitlab # [ 238.689783] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.836Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e7581519898dab319312721","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e7581519898dab319312721: done: 0.016099 sec","job_status":"done","queue_duration_s":1.566302,"scheduling_latency_s":1.566319,"redis_calls":13,"redis_duration_s":0.001906,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011277,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016099,"completed_at":"2026-10-04T19:07:51.836Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.730005] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12070846b81f2ea0b1c76cd0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12070846b81f2ea0b1c76cd0: start","job_status":"start","queue_duration_s":1.583956,"scheduling_latency_s":1.583973} gitlab # [ 238.742797] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"12070846b81f2ea0b1c76cd0","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-12070846b81f2ea0b1c76cd0: done: 0.01589 sec","job_status":"done","queue_duration_s":1.583956,"scheduling_latency_s":1.583973,"redis_calls":13,"redis_duration_s":0.001885,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001254,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01589,"completed_at":"2026-10-04T19:07:51.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.783308] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb75f348f2434df17a7c59bf","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb75f348f2434df17a7c59bf: start","job_status":"start","queue_duration_s":1.60157,"scheduling_latency_s":1.601589} gitlab # [ 238.796250] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eb75f348f2434df17a7c59bf","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eb75f348f2434df17a7c59bf: done: 0.015911 sec","job_status":"done","queue_duration_s":1.60157,"scheduling_latency_s":1.601589,"redis_calls":13,"redis_duration_s":0.00192,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000676,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001244,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015911,"completed_at":"2026-10-04T19:07:51.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.837176] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.873Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4579246027e2e1aab723f015","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4579246027e2e1aab723f015: start","job_status":"start","queue_duration_s":1.619072,"scheduling_latency_s":1.619091} gitlab # [ 238.857210] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4579246027e2e1aab723f015","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4579246027e2e1aab723f015: done: 0.016052 sec","job_status":"done","queue_duration_s":1.619072,"scheduling_latency_s":1.619091,"redis_calls":13,"redis_duration_s":0.002107,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000678,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001429,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016052,"completed_at":"2026-10-04T19:07:51.889Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.919618] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd09fcc5967f0fa103bf9828","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd09fcc5967f0fa103bf9828: start","job_status":"start","queue_duration_s":1.636337,"scheduling_latency_s":1.636353} gitlab # [ 238.936834] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.907Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd09fcc5967f0fa103bf9828","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd09fcc5967f0fa103bf9828: done: 0.01652 sec","job_status":"done","queue_duration_s":1.636337,"scheduling_latency_s":1.636353,"redis_calls":13,"redis_duration_s":0.002331,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000643,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001688,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01652,"completed_at":"2026-10-04T19:07:51.907Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 238.980078] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0d76fca0fb6e2852c1efda","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0d76fca0fb6e2852c1efda: start","job_status":"start","queue_duration_s":1.654014,"scheduling_latency_s":1.65403} gitlab # [ 238.993312] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.924Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b0d76fca0fb6e2852c1efda","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b0d76fca0fb6e2852c1efda: done: 0.016335 sec","job_status":"done","queue_duration_s":1.654014,"scheduling_latency_s":1.65403,"redis_calls":13,"redis_duration_s":0.00193,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000668,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001262,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010651,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016335,"completed_at":"2026-10-04T19:07:51.924Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.033905] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bed8325d2fe0c7a0ba199cb8","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bed8325d2fe0c7a0ba199cb8: start","job_status":"start","queue_duration_s":1.671544,"scheduling_latency_s":1.671561} gitlab # [ 239.046789] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.942Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bed8325d2fe0c7a0ba199cb8","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bed8325d2fe0c7a0ba199cb8: done: 0.016399 sec","job_status":"done","queue_duration_s":1.671544,"scheduling_latency_s":1.671561,"redis_calls":13,"redis_duration_s":0.002268,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000659,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001609,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016399,"completed_at":"2026-10-04T19:07:51.942Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.087286] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79e33a20dd5f67f9f7569cad","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79e33a20dd5f67f9f7569cad: start","job_status":"start","queue_duration_s":1.689193,"scheduling_latency_s":1.689211} gitlab # [ 239.101248] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79e33a20dd5f67f9f7569cad","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79e33a20dd5f67f9f7569cad: done: 0.016411 sec","job_status":"done","queue_duration_s":1.689193,"scheduling_latency_s":1.689211,"redis_calls":13,"redis_duration_s":0.002055,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001339,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010635,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016411,"completed_at":"2026-10-04T19:07:51.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.141863] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e13a9e124784e480ecfe625a","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e13a9e124784e480ecfe625a: start","job_status":"start","queue_duration_s":1.706783,"scheduling_latency_s":1.7068} gitlab # [ 239.154775] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e13a9e124784e480ecfe625a","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e13a9e124784e480ecfe625a: done: 0.015901 sec","job_status":"done","queue_duration_s":1.706783,"scheduling_latency_s":1.7068,"redis_calls":13,"redis_duration_s":0.00234,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001253,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015901,"completed_at":"2026-10-04T19:07:51.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.195322] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66b8f5c5b265aa12876da362","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66b8f5c5b265aa12876da362: start","job_status":"start","queue_duration_s":1.724376,"scheduling_latency_s":1.724394} gitlab # [ 239.208364] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:07:51.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66b8f5c5b265aa12876da362","created_at":"2026-10-04T19:07:50.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-8179ab76ff1d4881","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"8b8fc487b54fef19110c184c3cdea33e","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:07:50.254Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66b8f5c5b265aa12876da362: done: 0.015869 sec","job_status":"done","queue_duration_s":1.724376,"scheduling_latency_s":1.724394,"redis_calls":13,"redis_duration_s":0.001901,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000651,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015869,"completed_at":"2026-10-04T19:07:51.994Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.756579] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.677Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"02fd4a39d98457368b5ac087","created_at":"2026-10-04T19:08:01.661Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8919b2209e82a06d7979514f575e400","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.672Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-02fd4a39d98457368b5ac087: start","job_status":"start","queue_duration_s":0.005002,"scheduling_latency_s":0.005046} gitlab # [ 246.867227] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.788Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"02fd4a39d98457368b5ac087","created_at":"2026-10-04T19:08:01.661Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8919b2209e82a06d7979514f575e400","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.672Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-02fd4a39d98457368b5ac087: done: 0.111283 sec","job_status":"done","queue_duration_s":0.005002,"scheduling_latency_s":0.005046,"redis_calls":9,"redis_duration_s":0.021404,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.015933,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005471,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019791,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00503462,"exclusive_lock_hold_duration_s":0.00509854,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.111283,"completed_at":"2026-10-04T19:08:01.788Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002102,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 246.930089] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b8ca6e3df3b24f04e3c1adfc","created_at":"2026-10-04T19:08:01.705Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cffb7d6889fe7dcc7dafcb4c80e96b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:08:01.709Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b8ca6e3df3b24f04e3c1adfc: start","job_status":"start","queue_duration_s":0.086146,"scheduling_latency_s":0.086178} gitlab # [ 246.945702] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.809Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 246.949811] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b8ca6e3df3b24f04e3c1adfc","created_at":"2026-10-04T19:08:01.705Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1cffb7d6889fe7dcc7dafcb4c80e96b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:08:01.709Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b8ca6e3df3b24f04e3c1adfc: done: 0.037472 sec","job_status":"done","queue_duration_s":0.086146,"scheduling_latency_s":0.086178,"redis_calls":7,"redis_duration_s":0.003322,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00282,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000502,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037472,"completed_at":"2026-10-04T19:08:01.832Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.004728] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:01.926Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"67afce21e1c859ee3f0207b9","created_at":"2026-10-04T19:08:01.720Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33658fea45a63668097fb46f8c9bdac8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:08:01.724Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-67afce21e1c859ee3f0207b9: start","job_status":"start","queue_duration_s":0.110337,"scheduling_latency_s":0.110366} gitlab # [ 247.131030] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.052Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"67afce21e1c859ee3f0207b9","created_at":"2026-10-04T19:08:01.720Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33658fea45a63668097fb46f8c9bdac8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:08:01.724Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-67afce21e1c859ee3f0207b9: done: 0.217483 sec","job_status":"done","queue_duration_s":0.110337,"scheduling_latency_s":0.110366,"redis_calls":7,"redis_duration_s":0.011955,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003037,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027345,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.217483,"completed_at":"2026-10-04T19:08:02.052Z","load_balancing_strategy":"primary","db_duration_s":0.019036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.191769] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.074Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"854ed4669414db24ff74e61f","created_at":"2026-10-04T19:08:01.745Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bc1a3d9b91a224b0b79c547901ea85a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.754Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-854ed4669414db24ff74e61f: start","job_status":"start","queue_duration_s":0.320153,"scheduling_latency_s":0.320194} gitlab # [ 247.362441] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"854ed4669414db24ff74e61f","created_at":"2026-10-04T19:08:01.745Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3bc1a3d9b91a224b0b79c547901ea85a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.754Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-854ed4669414db24ff74e61f: done: 0.209252 sec","job_status":"done","queue_duration_s":0.320153,"scheduling_latency_s":0.320194,"redis_calls":9,"redis_duration_s":0.02043,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.018912,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001518,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.018,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035617,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00124596,"exclusive_lock_hold_duration_s":0.0636095,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.209252,"completed_at":"2026-10-04T19:08:02.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.06063,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.418927] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.285Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"23e92cfa6195ccf97e1dc003","created_at":"2026-10-04T19:08:01.774Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21aca6644fd0922f13821543a6596d52","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.780Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-23e92cfa6195ccf97e1dc003: start","job_status":"start","queue_duration_s":0.504602,"scheduling_latency_s":0.504628} gitlab # [ 247.607923] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"23e92cfa6195ccf97e1dc003","created_at":"2026-10-04T19:08:01.774Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21aca6644fd0922f13821543a6596d52","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.780Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-23e92cfa6195ccf97e1dc003: done: 0.242193 sec","job_status":"done","queue_duration_s":0.504602,"scheduling_latency_s":0.504628,"redis_calls":11,"redis_duration_s":0.002526,"redis_read_bytes":9,"redis_write_bytes":1262,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000372,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":275,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001067,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":130,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051948,"exclusive_lock_hold_duration_s":0.2187226,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.242193,"completed_at":"2026-10-04T19:08:02.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.05527,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.675905] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.529Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d2770c6c7db59ddc0f012114","created_at":"2026-10-04T19:08:01.790Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.794Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d2770c6c7db59ddc0f012114: start","job_status":"start","queue_duration_s":0.735013,"scheduling_latency_s":0.735039} gitlab # [ 247.692524] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.567Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"d2770c6c7db59ddc0f012114","created_at":"2026-10-04T19:08:01.790Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.794Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-d2770c6c7db59ddc0f012114: done: 0.038371 sec","job_status":"done","queue_duration_s":0.735013,"scheduling_latency_s":0.735039,"redis_calls":14,"redis_duration_s":0.003734,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000349,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002649,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000736,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038371,"completed_at":"2026-10-04T19:08:02.567Z","load_balancing_strategy":"primary","db_duration_s":0.003876,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.738306] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.569Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5e596e53cf2fffa414e5b114","created_at":"2026-10-04T19:08:01.823Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e3bb437f9f7267b2a6aea1062a3b515","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.830Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5e596e53cf2fffa414e5b114: start","job_status":"start","queue_duration_s":0.738895,"scheduling_latency_s":0.738916} gitlab # [ 247.753395] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.579Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 247.757040] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.584Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"5e596e53cf2fffa414e5b114","created_at":"2026-10-04T19:08:01.823Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e3bb437f9f7267b2a6aea1062a3b515","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.830Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-5e596e53cf2fffa414e5b114: done: 0.015663 sec","job_status":"done","queue_duration_s":0.738895,"scheduling_latency_s":0.738916,"redis_calls":7,"redis_duration_s":0.001341,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00037,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0098,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015663,"completed_at":"2026-10-04T19:08:02.584Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.798875] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.586Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1f02e46a932b1078340fb875","created_at":"2026-10-04T19:08:01.958Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e71aaf2776ae95954e3b1354533d79c9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.981Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1f02e46a932b1078340fb875: start","job_status":"start","queue_duration_s":0.604954,"scheduling_latency_s":0.604973} gitlab # [ 247.814151] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"1f02e46a932b1078340fb875","created_at":"2026-10-04T19:08:01.958Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e71aaf2776ae95954e3b1354533d79c9","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.981Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-1f02e46a932b1078340fb875: done: 0.02864 sec","job_status":"done","queue_duration_s":0.604954,"scheduling_latency_s":0.604973,"redis_calls":10,"redis_duration_s":0.001836,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000374,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00107,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000392,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029494,"duration_s":0.02864,"completed_at":"2026-10-04T19:08:02.614Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003631,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.858268] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.616Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0ed5c0baa79472cee4d9a84a","created_at":"2026-10-04T19:08:01.990Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a0d46d6b15259c90b085419aec854da","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.994Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0ed5c0baa79472cee4d9a84a: start","job_status":"start","queue_duration_s":0.62186,"scheduling_latency_s":0.621876} gitlab # [ 247.872984] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.631Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0ed5c0baa79472cee4d9a84a","created_at":"2026-10-04T19:08:01.990Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8a0d46d6b15259c90b085419aec854da","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:01.994Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0ed5c0baa79472cee4d9a84a: done: 0.014906 sec","job_status":"done","queue_duration_s":0.62186,"scheduling_latency_s":0.621876,"redis_calls":7,"redis_duration_s":0.00122,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000886,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014906,"completed_at":"2026-10-04T19:08:02.631Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000956,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 247.916124] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.632Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"07b0de624b09b84f0a34f503","created_at":"2026-10-04T19:08:02.000Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25b979c23bb0d937f80556212bf389aa","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.006Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-07b0de624b09b84f0a34f503: start","job_status":"start","queue_duration_s":0.626459,"scheduling_latency_s":0.626478} gitlab # [ 247.931192] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.653Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"07b0de624b09b84f0a34f503","created_at":"2026-10-04T19:08:02.000Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"25b979c23bb0d937f80556212bf389aa","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.006Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-07b0de624b09b84f0a34f503: done: 0.021033 sec","job_status":"done","queue_duration_s":0.626459,"scheduling_latency_s":0.626478,"redis_calls":10,"redis_duration_s":0.001974,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001133,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000841,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014146,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0052003,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021033,"completed_at":"2026-10-04T19:08:02.653Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002345,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 247.975551] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.654Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c402d15d26f0ffce8e8b1e41","created_at":"2026-10-04T19:08:02.016Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ab067fe0c00abecd08d5f3cb0abbf2e6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.029Z","job_size_bytes":3,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c402d15d26f0ffce8e8b1e41: start","job_status":"start","queue_duration_s":0.625004,"scheduling_latency_s":0.625022} gitlab # [ 247.990279] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.673Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"c402d15d26f0ffce8e8b1e41","created_at":"2026-10-04T19:08:02.016Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"ab067fe0c00abecd08d5f3cb0abbf2e6","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.029Z","job_size_bytes":3,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-c402d15d26f0ffce8e8b1e41: done: 0.018709 sec","job_status":"done","queue_duration_s":0.625004,"scheduling_latency_s":0.625022,"redis_calls":10,"redis_duration_s":0.001693,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028554,"duration_s":0.018709,"completed_at":"2026-10-04T19:08:02.673Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.033783] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.674Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"879ff7e97febb851d97a86de","created_at":"2026-10-04T19:08:02.044Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e6542882725efd9d2dcb1fe4d18aa9a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.057Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-879ff7e97febb851d97a86de: start","job_status":"start","queue_duration_s":0.617343,"scheduling_latency_s":0.61736} gitlab # [ 248.047498] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.694Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"879ff7e97febb851d97a86de","created_at":"2026-10-04T19:08:02.044Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6e6542882725efd9d2dcb1fe4d18aa9a","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.057Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-879ff7e97febb851d97a86de: done: 0.019282 sec","job_status":"done","queue_duration_s":0.617343,"scheduling_latency_s":0.61736,"redis_calls":12,"redis_duration_s":0.00228,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000423,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000354,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00117,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026508,"duration_s":0.019282,"completed_at":"2026-10-04T19:08:02.694Z","load_balancing_strategy":"primary","db_duration_s":0.000337,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.091766] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.695Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f91562c4492c6aed78155c0d","created_at":"2026-10-04T19:08:02.073Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4049d45f9dea0e7784029046f37095e1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.081Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f91562c4492c6aed78155c0d: start","job_status":"start","queue_duration_s":0.614328,"scheduling_latency_s":0.614345} gitlab # [ 248.107953] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"f91562c4492c6aed78155c0d","created_at":"2026-10-04T19:08:02.073Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4049d45f9dea0e7784029046f37095e1","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.081Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-f91562c4492c6aed78155c0d: done: 0.022578 sec","job_status":"done","queue_duration_s":0.614328,"scheduling_latency_s":0.614345,"redis_calls":7,"redis_duration_s":0.00136,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001008,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022578,"completed_at":"2026-10-04T19:08:02.717Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.150534] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.719Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"43660626aac18ca33fef9f26","created_at":"2026-10-04T19:08:02.084Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.088Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-43660626aac18ca33fef9f26: start","job_status":"start","queue_duration_s":0.630627,"scheduling_latency_s":0.630646} gitlab # [ 248.164585] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"43660626aac18ca33fef9f26","created_at":"2026-10-04T19:08:02.084Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.088Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-43660626aac18ca33fef9f26: done: 0.091895 sec","job_status":"done","queue_duration_s":0.630627,"scheduling_latency_s":0.630646,"redis_calls":60,"redis_duration_s":0.010971,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000665,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009807,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.083039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091895,"completed_at":"2026-10-04T19:08:02.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.207053] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.812Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"26a28d704f22c581d95b5675","created_at":"2026-10-04T19:08:02.094Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d48c1072b6eb14f97344192dafb2c5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.098Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-26a28d704f22c581d95b5675: start","job_status":"start","queue_duration_s":0.714194,"scheduling_latency_s":0.714214} gitlab # [ 248.222269] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.825Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"26a28d704f22c581d95b5675","created_at":"2026-10-04T19:08:02.094Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d48c1072b6eb14f97344192dafb2c5f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.098Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-26a28d704f22c581d95b5675: done: 0.013239 sec","job_status":"done","queue_duration_s":0.714194,"scheduling_latency_s":0.714214,"redis_calls":7,"redis_duration_s":0.001309,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000961,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008484,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013239,"completed_at":"2026-10-04T19:08:02.825Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.264448] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.827Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d33d78047e1e60fb9fde5043","created_at":"2026-10-04T19:08:02.116Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a55cbd2a6eb1aaf27ca7617a76d02a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.120Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d33d78047e1e60fb9fde5043: start","job_status":"start","queue_duration_s":0.707756,"scheduling_latency_s":0.707778} gitlab # [ 248.279637] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"d33d78047e1e60fb9fde5043","created_at":"2026-10-04T19:08:02.116Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a55cbd2a6eb1aaf27ca7617a76d02a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.120Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-d33d78047e1e60fb9fde5043: done: 0.013378 sec","job_status":"done","queue_duration_s":0.707756,"scheduling_latency_s":0.707778,"redis_calls":7,"redis_duration_s":0.00129,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000956,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008428,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013378,"completed_at":"2026-10-04T19:08:02.841Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.323590] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e895de1c9e153d55324d54f9","created_at":"2026-10-04T19:08:02.128Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8c198f8780bf2e5b9f25a66e2d3f43d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.130Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e895de1c9e153d55324d54f9: start","job_status":"start","queue_duration_s":0.711715,"scheduling_latency_s":0.711733} gitlab # [ 248.338935] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.878Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"e895de1c9e153d55324d54f9","created_at":"2026-10-04T19:08:02.128Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d8c198f8780bf2e5b9f25a66e2d3f43d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.130Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-e895de1c9e153d55324d54f9: done: 0.035595 sec","job_status":"done","queue_duration_s":0.711715,"scheduling_latency_s":0.711733,"redis_calls":9,"redis_duration_s":0.001774,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007097,"exclusive_lock_hold_duration_s":0.01471956,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.035595,"completed_at":"2026-10-04T19:08:02.878Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0137,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.383779] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.879Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ad2e3a8eb21d837676b9cf03","created_at":"2026-10-04T19:08:02.140Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91a773c0fd1142ed8970feef252f67ed","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.147Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ad2e3a8eb21d837676b9cf03: start","job_status":"start","queue_duration_s":0.731762,"scheduling_latency_s":0.731781} gitlab # [ 248.398991] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.904Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ad2e3a8eb21d837676b9cf03","created_at":"2026-10-04T19:08:02.140Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"91a773c0fd1142ed8970feef252f67ed","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.147Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ad2e3a8eb21d837676b9cf03: done: 0.024617 sec","job_status":"done","queue_duration_s":0.731762,"scheduling_latency_s":0.731781,"redis_calls":12,"redis_duration_s":0.002211,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0004,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000828,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00059766,"exclusive_lock_hold_duration_s":0.0054299199999999995,"duration_s":0.024617,"completed_at":"2026-10-04T19:08:02.904Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.443149] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0876a72353c2defb4410933d","created_at":"2026-10-04T19:08:02.164Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f03b38983ac65bbf4a4365aac87e1331","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.166Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0876a72353c2defb4410933d: start","job_status":"start","queue_duration_s":0.738344,"scheduling_latency_s":0.738363} gitlab # [ 248.457420] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.921Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0876a72353c2defb4410933d","created_at":"2026-10-04T19:08:02.164Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f03b38983ac65bbf4a4365aac87e1331","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.166Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0876a72353c2defb4410933d: done: 0.016681 sec","job_status":"done","queue_duration_s":0.738344,"scheduling_latency_s":0.738363,"redis_calls":9,"redis_duration_s":0.001359,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000724,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046656,"exclusive_lock_hold_duration_s":0.0031785,"duration_s":0.016681,"completed_at":"2026-10-04T19:08:02.921Z","load_balancing_strategy":"primary","db_duration_s":0.00093,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.498803] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.923Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"677e3e97371317116014640f","created_at":"2026-10-04T19:08:02.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"162df87d1534b4d3cb87c1ab5726b1f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.173Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-677e3e97371317116014640f: start","job_status":"start","queue_duration_s":0.749927,"scheduling_latency_s":0.749949} gitlab # [ 248.514369] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.945Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"677e3e97371317116014640f","created_at":"2026-10-04T19:08:02.170Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"162df87d1534b4d3cb87c1ab5726b1f9","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.173Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-677e3e97371317116014640f: done: 0.022704 sec","job_status":"done","queue_duration_s":0.749927,"scheduling_latency_s":0.749949,"redis_calls":7,"redis_duration_s":0.001027,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.022704,"completed_at":"2026-10-04T19:08:02.945Z","load_balancing_strategy":"primary","db_duration_s":0.008179,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.557073] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:02.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"84a4f95f65836aa1dada32ad","created_at":"2026-10-04T19:08:02.177Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddc7f03f622665773d74f6aef4a982e4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.180Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-84a4f95f65836aa1dada32ad: start","job_status":"start","queue_duration_s":0.767207,"scheduling_latency_s":0.767225} gitlab # [ 248.571664] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.276Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"84a4f95f65836aa1dada32ad","created_at":"2026-10-04T19:08:02.177Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddc7f03f622665773d74f6aef4a982e4","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.180Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-84a4f95f65836aa1dada32ad: done: 0.329667 sec","job_status":"done","queue_duration_s":0.767207,"scheduling_latency_s":0.767225,"redis_calls":562,"redis_duration_s":0.058145999999999996,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.057404,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.307967,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.329667,"completed_at":"2026-10-04T19:08:03.276Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.614170] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.278Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e00da0a9fe56a8295349c3c3","created_at":"2026-10-04T19:08:02.204Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de1fd6bd1a1a2047e002e89206d6fd4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.210Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e00da0a9fe56a8295349c3c3: start","job_status":"start","queue_duration_s":1.067654,"scheduling_latency_s":1.067683} gitlab # [ 248.629540] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.286Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 248.633079] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"e00da0a9fe56a8295349c3c3","created_at":"2026-10-04T19:08:02.204Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4de1fd6bd1a1a2047e002e89206d6fd4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.210Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-e00da0a9fe56a8295349c3c3: done: 0.014619 sec","job_status":"done","queue_duration_s":1.067654,"scheduling_latency_s":1.067683,"redis_calls":7,"redis_duration_s":0.0010509999999999999,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000752,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000299,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014619,"completed_at":"2026-10-04T19:08:03.292Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.675366] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.294Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"80742ee5e5093dbc66f1ed30","created_at":"2026-10-04T19:08:02.219Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7305acfc7067a2eaed16e9a72c8a9bdd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.223Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-80742ee5e5093dbc66f1ed30: start","job_status":"start","queue_duration_s":1.071333,"scheduling_latency_s":1.071353} gitlab # [ 248.690811] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.302Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 248.694833] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.307Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"80742ee5e5093dbc66f1ed30","created_at":"2026-10-04T19:08:02.219Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7305acfc7067a2eaed16e9a72c8a9bdd","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.223Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-80742ee5e5093dbc66f1ed30: done: 0.013008 sec","job_status":"done","queue_duration_s":1.071333,"scheduling_latency_s":1.071353,"redis_calls":7,"redis_duration_s":0.000973,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008359,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013008,"completed_at":"2026-10-04T19:08:03.307Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.738613] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.308Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"784718d7ed8c6d0f52f7697c","created_at":"2026-10-04T19:08:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db36f043b95f82d6efc52cff43e2534","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.231Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-784718d7ed8c6d0f52f7697c: start","job_status":"start","queue_duration_s":1.077808,"scheduling_latency_s":1.077827} gitlab # [ 248.753594] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.316Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 248.757636] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.321Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"784718d7ed8c6d0f52f7697c","created_at":"2026-10-04T19:08:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9db36f043b95f82d6efc52cff43e2534","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.231Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-784718d7ed8c6d0f52f7697c: done: 0.012611 sec","job_status":"done","queue_duration_s":1.077808,"scheduling_latency_s":1.077827,"redis_calls":7,"redis_duration_s":0.0009910000000000001,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000302,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012611,"completed_at":"2026-10-04T19:08:03.321Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.799814] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.322Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f24013e36d1de30ae1887d3d","created_at":"2026-10-04T19:08:02.234Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d67337f958d5704badcb439742b766d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:08:02.236Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f24013e36d1de30ae1887d3d: start","job_status":"start","queue_duration_s":1.085778,"scheduling_latency_s":1.085798} gitlab # [ 248.814035] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.330Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 248.817511] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f24013e36d1de30ae1887d3d","created_at":"2026-10-04T19:08:02.234Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d67337f958d5704badcb439742b766d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:08:02.236Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f24013e36d1de30ae1887d3d: done: 0.012649 sec","job_status":"done","queue_duration_s":1.085778,"scheduling_latency_s":1.085798,"redis_calls":7,"redis_duration_s":0.000975,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000703,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000272,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012649,"completed_at":"2026-10-04T19:08:03.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.858550] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.336Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acc08df142ace696b4433b09","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acc08df142ace696b4433b09: start","job_status":"start","queue_duration_s":0.775581,"scheduling_latency_s":0.775599} gitlab # [ 248.871376] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.348Z","message":"No existing merge request to be cleaned up."} gitlab # [ 248.873196] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.357Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acc08df142ace696b4433b09","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acc08df142ace696b4433b09: done: 0.021107 sec","job_status":"done","queue_duration_s":0.775581,"scheduling_latency_s":0.775599,"redis_calls":10,"redis_duration_s":0.001454,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000605,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000849,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021107,"completed_at":"2026-10-04T19:08:03.357Z","load_balancing_strategy":"primary","db_duration_s":0.002579,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.912895] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.358Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a319e1a2664533a788e48d1f","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a319e1a2664533a788e48d1f: start","job_status":"start","queue_duration_s":0.79792,"scheduling_latency_s":0.797938} gitlab # [ 248.926761] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.369Z","message":"No existing merge request to be cleaned up."} gitlab # [ 248.928545] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.378Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a319e1a2664533a788e48d1f","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a319e1a2664533a788e48d1f: done: 0.019342 sec","job_status":"done","queue_duration_s":0.79792,"scheduling_latency_s":0.797938,"redis_calls":10,"redis_duration_s":0.001521,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000639,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000882,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019342,"completed_at":"2026-10-04T19:08:03.378Z","load_balancing_strategy":"primary","db_duration_s":0.002014,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 248.968451] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.379Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b3ceb41a9aa5550add4bf20a","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b3ceb41a9aa5550add4bf20a: start","job_status":"start","queue_duration_s":0.818493,"scheduling_latency_s":0.818512} gitlab # [ 248.981108] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.390Z","message":"No existing merge request to be cleaned up."} gitlab # [ 248.982775] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.398Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b3ceb41a9aa5550add4bf20a","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b3ceb41a9aa5550add4bf20a: done: 0.019108 sec","job_status":"done","queue_duration_s":0.818493,"scheduling_latency_s":0.818512,"redis_calls":10,"redis_duration_s":0.0017950000000000002,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019108,"completed_at":"2026-10-04T19:08:03.398Z","load_balancing_strategy":"primary","db_duration_s":0.001549,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.023043] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.399Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"559a852a8736c72096d0b0db","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-559a852a8736c72096d0b0db: start","job_status":"start","queue_duration_s":0.83875,"scheduling_latency_s":0.838768} gitlab # [ 249.035868] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.409Z","message":"No existing merge request to be cleaned up."} gitlab # [ 249.037709] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.417Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"559a852a8736c72096d0b0db","created_at":"2026-10-04T19:08:02.549Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-3abd33eb4b47461c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"65e390f4d383fe2276c8cd5c5363f806","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.560Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-559a852a8736c72096d0b0db: done: 0.017921 sec","job_status":"done","queue_duration_s":0.83875,"scheduling_latency_s":0.838768,"redis_calls":10,"redis_duration_s":0.00136,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012708,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017921,"completed_at":"2026-10-04T19:08:03.417Z","load_balancing_strategy":"primary","db_duration_s":0.001362,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.077687] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bcba185264ce627cc92afa8","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bcba185264ce627cc92afa8: start","job_status":"start","queue_duration_s":0.614878,"scheduling_latency_s":0.614897} gitlab # [ 249.090479] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.433Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bcba185264ce627cc92afa8","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bcba185264ce627cc92afa8: done: 0.014956 sec","job_status":"done","queue_duration_s":0.614878,"scheduling_latency_s":0.614897,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014956,"completed_at":"2026-10-04T19:08:03.433Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.132459] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.434Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"927db43505986934787ad530","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-927db43505986934787ad530: start","job_status":"start","queue_duration_s":0.630995,"scheduling_latency_s":0.631014} gitlab # [ 249.145285] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.449Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"927db43505986934787ad530","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-927db43505986934787ad530: done: 0.01454 sec","job_status":"done","queue_duration_s":0.630995,"scheduling_latency_s":0.631014,"redis_calls":13,"redis_duration_s":0.0015470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001007,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010021,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01454,"completed_at":"2026-10-04T19:08:03.449Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.185819] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84e28ae792992ec8c8a44f01","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84e28ae792992ec8c8a44f01: start","job_status":"start","queue_duration_s":0.646759,"scheduling_latency_s":0.646778} gitlab # [ 249.198729] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.465Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"84e28ae792992ec8c8a44f01","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-84e28ae792992ec8c8a44f01: done: 0.014597 sec","job_status":"done","queue_duration_s":0.646759,"scheduling_latency_s":0.646778,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010167,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014597,"completed_at":"2026-10-04T19:08:03.465Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.239626] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6818c1fd04cea476d0860c","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6818c1fd04cea476d0860c: start","job_status":"start","queue_duration_s":0.662545,"scheduling_latency_s":0.662565} gitlab # [ 249.252553] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.480Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd6818c1fd04cea476d0860c","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd6818c1fd04cea476d0860c: done: 0.014312 sec","job_status":"done","queue_duration_s":0.662545,"scheduling_latency_s":0.662565,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014312,"completed_at":"2026-10-04T19:08:03.480Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.293032] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.481Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a74b3348e2503cc90b749bd","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a74b3348e2503cc90b749bd: start","job_status":"start","queue_duration_s":0.677942,"scheduling_latency_s":0.67796} gitlab # [ 249.305997] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.496Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9a74b3348e2503cc90b749bd","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9a74b3348e2503cc90b749bd: done: 0.014683 sec","job_status":"done","queue_duration_s":0.677942,"scheduling_latency_s":0.67796,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014683,"completed_at":"2026-10-04T19:08:03.496Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.347813] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68d6462224bb5a29ae61667d","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68d6462224bb5a29ae61667d: start","job_status":"start","queue_duration_s":0.693708,"scheduling_latency_s":0.693725} gitlab # [ 249.361006] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.512Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68d6462224bb5a29ae61667d","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68d6462224bb5a29ae61667d: done: 0.014627 sec","job_status":"done","queue_duration_s":0.693708,"scheduling_latency_s":0.693725,"redis_calls":13,"redis_duration_s":0.001577,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000616,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014627,"completed_at":"2026-10-04T19:08:03.512Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.401367] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0db1b7b874b16ebb9618fbd5","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0db1b7b874b16ebb9618fbd5: start","job_status":"start","queue_duration_s":0.709436,"scheduling_latency_s":0.709453} gitlab # [ 249.414214] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.530Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0db1b7b874b16ebb9618fbd5","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0db1b7b874b16ebb9618fbd5: done: 0.017496 sec","job_status":"done","queue_duration_s":0.709436,"scheduling_latency_s":0.709453,"redis_calls":13,"redis_duration_s":0.0015439999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010279,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017496,"completed_at":"2026-10-04T19:08:03.530Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.455083] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70e959386327b344fb8051a","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70e959386327b344fb8051a: start","job_status":"start","queue_duration_s":0.728053,"scheduling_latency_s":0.728071} gitlab # [ 249.468121] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.546Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e70e959386327b344fb8051a","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e70e959386327b344fb8051a: done: 0.014681 sec","job_status":"done","queue_duration_s":0.728053,"scheduling_latency_s":0.728071,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-10-04T19:08:03.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.508805] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61e85d9753c315aabd30c9b2","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61e85d9753c315aabd30c9b2: start","job_status":"start","queue_duration_s":0.743904,"scheduling_latency_s":0.743922} gitlab # [ 249.521537] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.562Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61e85d9753c315aabd30c9b2","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61e85d9753c315aabd30c9b2: done: 0.014412 sec","job_status":"done","queue_duration_s":0.743904,"scheduling_latency_s":0.743922,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014412,"completed_at":"2026-10-04T19:08:03.562Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.563830] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.563Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c20056a8a33633fbfd7ee3aa","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c20056a8a33633fbfd7ee3aa: start","job_status":"start","queue_duration_s":0.759481,"scheduling_latency_s":0.7595} gitlab # [ 249.583947] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.578Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c20056a8a33633fbfd7ee3aa","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c20056a8a33633fbfd7ee3aa: done: 0.015574 sec","job_status":"done","queue_duration_s":0.759481,"scheduling_latency_s":0.7595,"redis_calls":13,"redis_duration_s":0.0016510000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001051,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015574,"completed_at":"2026-10-04T19:08:03.578Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.637324] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.580Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afdcf1bee7d28cfe920c5213","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afdcf1bee7d28cfe920c5213: start","job_status":"start","queue_duration_s":0.776162,"scheduling_latency_s":0.77618} gitlab # [ 249.650073] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.595Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afdcf1bee7d28cfe920c5213","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afdcf1bee7d28cfe920c5213: done: 0.015433 sec","job_status":"done","queue_duration_s":0.776162,"scheduling_latency_s":0.77618,"redis_calls":13,"redis_duration_s":0.0016409999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000627,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015433,"completed_at":"2026-10-04T19:08:03.595Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.691115] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.596Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a93e360459808f870b1c6944","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a93e360459808f870b1c6944: start","job_status":"start","queue_duration_s":0.792975,"scheduling_latency_s":0.792993} gitlab # [ 249.704144] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.611Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a93e360459808f870b1c6944","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a93e360459808f870b1c6944: done: 0.014529 sec","job_status":"done","queue_duration_s":0.792975,"scheduling_latency_s":0.792993,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000983,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014529,"completed_at":"2026-10-04T19:08:03.611Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.746040] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.612Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50805850249eeb0ecb24a41f","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50805850249eeb0ecb24a41f: start","job_status":"start","queue_duration_s":0.808578,"scheduling_latency_s":0.808597} gitlab # [ 249.759234] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.626Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"50805850249eeb0ecb24a41f","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-50805850249eeb0ecb24a41f: done: 0.01427 sec","job_status":"done","queue_duration_s":0.808578,"scheduling_latency_s":0.808597,"redis_calls":13,"redis_duration_s":0.001458,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009876,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01427,"completed_at":"2026-10-04T19:08:03.626Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.799971] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.627Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9844ae2bb2e9a84a75e132f","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9844ae2bb2e9a84a75e132f: start","job_status":"start","queue_duration_s":0.823978,"scheduling_latency_s":0.823996} gitlab # [ 249.813052] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.642Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a9844ae2bb2e9a84a75e132f","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a9844ae2bb2e9a84a75e132f: done: 0.014334 sec","job_status":"done","queue_duration_s":0.823978,"scheduling_latency_s":0.823996,"redis_calls":13,"redis_duration_s":0.0014589999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009936,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014334,"completed_at":"2026-10-04T19:08:03.642Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.853562] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.643Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"761d0073c8ecfe6d3caa1500","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-761d0073c8ecfe6d3caa1500: start","job_status":"start","queue_duration_s":0.839609,"scheduling_latency_s":0.839627} gitlab # [ 249.866447] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.657Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"761d0073c8ecfe6d3caa1500","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-761d0073c8ecfe6d3caa1500: done: 0.014406 sec","job_status":"done","queue_duration_s":0.839609,"scheduling_latency_s":0.839627,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014406,"completed_at":"2026-10-04T19:08:03.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.907076] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"691733268bbed4220d5183e0","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-691733268bbed4220d5183e0: start","job_status":"start","queue_duration_s":0.855187,"scheduling_latency_s":0.855206} gitlab # [ 249.920207] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.673Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"691733268bbed4220d5183e0","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-691733268bbed4220d5183e0: done: 0.014241 sec","job_status":"done","queue_duration_s":0.855187,"scheduling_latency_s":0.855206,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014241,"completed_at":"2026-10-04T19:08:03.673Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 249.961826] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dc16153255fcf343415ae18","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dc16153255fcf343415ae18: start","job_status":"start","queue_duration_s":0.870557,"scheduling_latency_s":0.870575} gitlab # [ 249.981812] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4dc16153255fcf343415ae18","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4dc16153255fcf343415ae18: done: 0.014234 sec","job_status":"done","queue_duration_s":0.870557,"scheduling_latency_s":0.870575,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014234,"completed_at":"2026-10-04T19:08:03.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.040241] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66499c8d6de8c29a6791809","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66499c8d6de8c29a6791809: start","job_status":"start","queue_duration_s":0.885869,"scheduling_latency_s":0.885887} gitlab # [ 250.054492] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e66499c8d6de8c29a6791809","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e66499c8d6de8c29a6791809: done: 0.016084 sec","job_status":"done","queue_duration_s":0.885869,"scheduling_latency_s":0.885887,"redis_calls":13,"redis_duration_s":0.0016970000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001149,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010592,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016084,"completed_at":"2026-10-04T19:08:03.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.095446] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99a106a36dec97d1e9fc5d45","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99a106a36dec97d1e9fc5d45: start","job_status":"start","queue_duration_s":0.903218,"scheduling_latency_s":0.903237} gitlab # [ 250.108331] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99a106a36dec97d1e9fc5d45","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99a106a36dec97d1e9fc5d45: done: 0.014299 sec","job_status":"done","queue_duration_s":0.903218,"scheduling_latency_s":0.903237,"redis_calls":13,"redis_duration_s":0.0014550000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014299,"completed_at":"2026-10-04T19:08:03.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.148742] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06daf5622c30ee3e19411ea9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06daf5622c30ee3e19411ea9: start","job_status":"start","queue_duration_s":0.918659,"scheduling_latency_s":0.918677} gitlab # [ 250.162425] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.736Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06daf5622c30ee3e19411ea9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06daf5622c30ee3e19411ea9: done: 0.014226 sec","job_status":"done","queue_duration_s":0.918659,"scheduling_latency_s":0.918677,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000525,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000925,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014226,"completed_at":"2026-10-04T19:08:03.736Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.202818] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d79e1c8660a604fbeea170a6","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d79e1c8660a604fbeea170a6: start","job_status":"start","queue_duration_s":0.933964,"scheduling_latency_s":0.933982} gitlab # [ 250.215756] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d79e1c8660a604fbeea170a6","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d79e1c8660a604fbeea170a6: done: 0.015303 sec","job_status":"done","queue_duration_s":0.933964,"scheduling_latency_s":0.933982,"redis_calls":13,"redis_duration_s":0.001856,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015303,"completed_at":"2026-10-04T19:08:03.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.256388] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f61169b0057ce44480e8e8","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f61169b0057ce44480e8e8: start","job_status":"start","queue_duration_s":0.95037,"scheduling_latency_s":0.950388} gitlab # [ 250.269054] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.768Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6f61169b0057ce44480e8e8","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6f61169b0057ce44480e8e8: done: 0.014492 sec","job_status":"done","queue_duration_s":0.95037,"scheduling_latency_s":0.950388,"redis_calls":13,"redis_duration_s":0.001568,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001041,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009987,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014492,"completed_at":"2026-10-04T19:08:03.768Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.309215] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50e9d0e67acfa0be77f7481","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50e9d0e67acfa0be77f7481: start","job_status":"start","queue_duration_s":0.965937,"scheduling_latency_s":0.965955} gitlab # [ 250.322007] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50e9d0e67acfa0be77f7481","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50e9d0e67acfa0be77f7481: done: 0.014351 sec","job_status":"done","queue_duration_s":0.965937,"scheduling_latency_s":0.965955,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001016,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014351,"completed_at":"2026-10-04T19:08:03.784Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.363169] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae0c86253a0c3aec6e703708","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae0c86253a0c3aec6e703708: start","job_status":"start","queue_duration_s":0.981383,"scheduling_latency_s":0.9814} gitlab # [ 250.376100] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ae0c86253a0c3aec6e703708","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ae0c86253a0c3aec6e703708: done: 0.015503 sec","job_status":"done","queue_duration_s":0.981383,"scheduling_latency_s":0.9814,"redis_calls":13,"redis_duration_s":0.0017850000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001219,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015503,"completed_at":"2026-10-04T19:08:03.800Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.416574] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2236db849f9ea2b4a5aafaf4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2236db849f9ea2b4a5aafaf4: start","job_status":"start","queue_duration_s":0.998057,"scheduling_latency_s":0.998074} gitlab # [ 250.429262] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.816Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2236db849f9ea2b4a5aafaf4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2236db849f9ea2b4a5aafaf4: done: 0.014201 sec","job_status":"done","queue_duration_s":0.998057,"scheduling_latency_s":0.998074,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014201,"completed_at":"2026-10-04T19:08:03.816Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.469388] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e7ebf409e23090eecb41bb","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e7ebf409e23090eecb41bb: start","job_status":"start","queue_duration_s":1.013399,"scheduling_latency_s":1.013418} gitlab # [ 250.482099] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56e7ebf409e23090eecb41bb","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56e7ebf409e23090eecb41bb: done: 0.014276 sec","job_status":"done","queue_duration_s":1.013399,"scheduling_latency_s":1.013418,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014276,"completed_at":"2026-10-04T19:08:03.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.522468] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.832Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3663e7a0d15dc363d9bad19d","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3663e7a0d15dc363d9bad19d: start","job_status":"start","queue_duration_s":1.028771,"scheduling_latency_s":1.028788} gitlab # [ 250.535342] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.847Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3663e7a0d15dc363d9bad19d","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3663e7a0d15dc363d9bad19d: done: 0.014417 sec","job_status":"done","queue_duration_s":1.028771,"scheduling_latency_s":1.028788,"redis_calls":13,"redis_duration_s":0.0014850000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014417,"completed_at":"2026-10-04T19:08:03.846Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.576941] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.848Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97f0db8ccfd9a6deeabffb69","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97f0db8ccfd9a6deeabffb69: start","job_status":"start","queue_duration_s":1.04433,"scheduling_latency_s":1.04435} gitlab # [ 250.589653] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97f0db8ccfd9a6deeabffb69","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97f0db8ccfd9a6deeabffb69: done: 0.014324 sec","job_status":"done","queue_duration_s":1.04433,"scheduling_latency_s":1.04435,"redis_calls":13,"redis_duration_s":0.001433,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009723,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014324,"completed_at":"2026-10-04T19:08:03.862Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.630128] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8e5a7d3c9095b48cf1cad4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8e5a7d3c9095b48cf1cad4: start","job_status":"start","queue_duration_s":1.059725,"scheduling_latency_s":1.059742} gitlab # [ 250.642931] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.877Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c8e5a7d3c9095b48cf1cad4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c8e5a7d3c9095b48cf1cad4: done: 0.01408 sec","job_status":"done","queue_duration_s":1.059725,"scheduling_latency_s":1.059742,"redis_calls":13,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009852,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01408,"completed_at":"2026-10-04T19:08:03.877Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.683707] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5124c148161602492be39859","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5124c148161602492be39859: start","job_status":"start","queue_duration_s":1.074951,"scheduling_latency_s":1.074969} gitlab # [ 250.696562] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.892Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5124c148161602492be39859","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5124c148161602492be39859: done: 0.014146 sec","job_status":"done","queue_duration_s":1.074951,"scheduling_latency_s":1.074969,"redis_calls":13,"redis_duration_s":0.0014429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014146,"completed_at":"2026-10-04T19:08:03.892Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.737168] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.894Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972fd202ca8605ba5ed50f19","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972fd202ca8605ba5ed50f19: start","job_status":"start","queue_duration_s":1.090206,"scheduling_latency_s":1.090224} gitlab # [ 250.750004] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"972fd202ca8605ba5ed50f19","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-972fd202ca8605ba5ed50f19: done: 0.014919 sec","job_status":"done","queue_duration_s":1.090206,"scheduling_latency_s":1.090224,"redis_calls":13,"redis_duration_s":0.001944,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001354,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014919,"completed_at":"2026-10-04T19:08:03.908Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.791558] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.910Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81c59c00e779eb337cd050b5","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81c59c00e779eb337cd050b5: start","job_status":"start","queue_duration_s":1.106233,"scheduling_latency_s":1.10625} gitlab # [ 250.804658] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81c59c00e779eb337cd050b5","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81c59c00e779eb337cd050b5: done: 0.013925 sec","job_status":"done","queue_duration_s":1.106233,"scheduling_latency_s":1.10625,"redis_calls":13,"redis_duration_s":0.0017920000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001157,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013925,"completed_at":"2026-10-04T19:08:03.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.845634] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a4216bfd8de426e93140759","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a4216bfd8de426e93140759: start","job_status":"start","queue_duration_s":1.121307,"scheduling_latency_s":1.121323} gitlab # [ 250.865742] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a4216bfd8de426e93140759","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a4216bfd8de426e93140759: done: 0.013826 sec","job_status":"done","queue_duration_s":1.121307,"scheduling_latency_s":1.121323,"redis_calls":13,"redis_duration_s":0.001794,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001145,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013826,"completed_at":"2026-10-04T19:08:03.938Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.924047] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22759ee3081b169ff1e037e3","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22759ee3081b169ff1e037e3: start","job_status":"start","queue_duration_s":1.136369,"scheduling_latency_s":1.136386} gitlab # [ 250.938315] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.954Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"22759ee3081b169ff1e037e3","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-22759ee3081b169ff1e037e3: done: 0.01438 sec","job_status":"done","queue_duration_s":1.136369,"scheduling_latency_s":1.136386,"redis_calls":13,"redis_duration_s":0.001935,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001251,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009624,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01438,"completed_at":"2026-10-04T19:08:03.954Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 250.979077] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c659eaf8e7d83c11126e10","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c659eaf8e7d83c11126e10: start","job_status":"start","queue_duration_s":1.151958,"scheduling_latency_s":1.151975} gitlab # [ 250.992891] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.969Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"06c659eaf8e7d83c11126e10","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-06c659eaf8e7d83c11126e10: done: 0.013929 sec","job_status":"done","queue_duration_s":1.151958,"scheduling_latency_s":1.151975,"redis_calls":13,"redis_duration_s":0.001826,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000624,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0093,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013929,"completed_at":"2026-10-04T19:08:03.969Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.032912] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc7613006852c3f7ea7f97c9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc7613006852c3f7ea7f97c9: start","job_status":"start","queue_duration_s":1.167113,"scheduling_latency_s":1.167129} gitlab # [ 251.045575] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.987Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc7613006852c3f7ea7f97c9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc7613006852c3f7ea7f97c9: done: 0.016492 sec","job_status":"done","queue_duration_s":1.167113,"scheduling_latency_s":1.167129,"redis_calls":13,"redis_duration_s":0.004207,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000636,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003571,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009551,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016492,"completed_at":"2026-10-04T19:08:03.987Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.085921] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:03.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"032562e8722246b2e7098998","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-032562e8722246b2e7098998: start","job_status":"start","queue_duration_s":1.184907,"scheduling_latency_s":1.184925} gitlab # [ 251.098767] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"032562e8722246b2e7098998","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-032562e8722246b2e7098998: done: 0.014398 sec","job_status":"done","queue_duration_s":1.184907,"scheduling_latency_s":1.184925,"redis_calls":13,"redis_duration_s":0.001902,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009534,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014398,"completed_at":"2026-10-04T19:08:04.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.139326] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.073Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07b69a4989831f198ea2b513","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07b69a4989831f198ea2b513: start","job_status":"start","queue_duration_s":1.269491,"scheduling_latency_s":1.269515} gitlab # [ 251.152354] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07b69a4989831f198ea2b513","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07b69a4989831f198ea2b513: done: 0.016095 sec","job_status":"done","queue_duration_s":1.269491,"scheduling_latency_s":1.269515,"redis_calls":13,"redis_duration_s":0.0020080000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000758,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00125,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016095,"completed_at":"2026-10-04T19:08:04.089Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.193998] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.090Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d994857ce91c03d0e6f2593","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d994857ce91c03d0e6f2593: start","job_status":"start","queue_duration_s":1.286796,"scheduling_latency_s":1.286813} gitlab # [ 251.206774] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7d994857ce91c03d0e6f2593","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7d994857ce91c03d0e6f2593: done: 0.01537 sec","job_status":"done","queue_duration_s":1.286796,"scheduling_latency_s":1.286813,"redis_calls":13,"redis_duration_s":0.002176,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000649,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001527,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01537,"completed_at":"2026-10-04T19:08:04.105Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.247353] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.107Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89dfe8cab1a5498d43a763a7","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89dfe8cab1a5498d43a763a7: start","job_status":"start","queue_duration_s":1.303361,"scheduling_latency_s":1.303378} gitlab # [ 251.260319] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"89dfe8cab1a5498d43a763a7","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-89dfe8cab1a5498d43a763a7: done: 0.014585 sec","job_status":"done","queue_duration_s":1.303361,"scheduling_latency_s":1.303378,"redis_calls":13,"redis_duration_s":0.001895,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001235,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009974,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014585,"completed_at":"2026-10-04T19:08:04.121Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.300758] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68693a1b32b2d7196a61dca2","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68693a1b32b2d7196a61dca2: start","job_status":"start","queue_duration_s":1.31919,"scheduling_latency_s":1.3199} gitlab # [ 251.313430] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.138Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68693a1b32b2d7196a61dca2","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68693a1b32b2d7196a61dca2: done: 0.015424 sec","job_status":"done","queue_duration_s":1.31919,"scheduling_latency_s":1.3199,"redis_calls":13,"redis_duration_s":0.002463,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001245,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015424,"completed_at":"2026-10-04T19:08:04.138Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.353782] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.139Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"047064485fa7d1bad976b153","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-047064485fa7d1bad976b153: start","job_status":"start","queue_duration_s":1.335773,"scheduling_latency_s":1.335788} gitlab # [ 251.366684] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"047064485fa7d1bad976b153","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-047064485fa7d1bad976b153: done: 0.01482 sec","job_status":"done","queue_duration_s":1.335773,"scheduling_latency_s":1.335788,"redis_calls":13,"redis_duration_s":0.001865,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01482,"completed_at":"2026-10-04T19:08:04.154Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.408544] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.155Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99b4855e97025f3770e4a83a","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99b4855e97025f3770e4a83a: start","job_status":"start","queue_duration_s":1.35179,"scheduling_latency_s":1.351806} gitlab # [ 251.421358] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.170Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99b4855e97025f3770e4a83a","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99b4855e97025f3770e4a83a: done: 0.014973 sec","job_status":"done","queue_duration_s":1.35179,"scheduling_latency_s":1.351806,"redis_calls":13,"redis_duration_s":0.0019649999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000673,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014973,"completed_at":"2026-10-04T19:08:04.170Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.461590] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ecb1788675fe1f06cd1ba59","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ecb1788675fe1f06cd1ba59: start","job_status":"start","queue_duration_s":1.367893,"scheduling_latency_s":1.367908} gitlab # [ 251.474415] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ecb1788675fe1f06cd1ba59","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ecb1788675fe1f06cd1ba59: done: 0.015526 sec","job_status":"done","queue_duration_s":1.367893,"scheduling_latency_s":1.367908,"redis_calls":13,"redis_duration_s":0.001809,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000629,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00118,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010062,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015526,"completed_at":"2026-10-04T19:08:04.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.515182] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86c1a7eee0d59b3dcfe366df","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86c1a7eee0d59b3dcfe366df: start","job_status":"start","queue_duration_s":1.384543,"scheduling_latency_s":1.38456} gitlab # [ 251.528189] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86c1a7eee0d59b3dcfe366df","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86c1a7eee0d59b3dcfe366df: done: 0.015214 sec","job_status":"done","queue_duration_s":1.384543,"scheduling_latency_s":1.38456,"redis_calls":13,"redis_duration_s":0.002114,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001277,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015214,"completed_at":"2026-10-04T19:08:04.203Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.568643] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.204Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3304ca2cfb3ba3645645a1b9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3304ca2cfb3ba3645645a1b9: start","job_status":"start","queue_duration_s":1.400936,"scheduling_latency_s":1.400951} gitlab # [ 251.581615] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3304ca2cfb3ba3645645a1b9","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3304ca2cfb3ba3645645a1b9: done: 0.014577 sec","job_status":"done","queue_duration_s":1.400936,"scheduling_latency_s":1.400951,"redis_calls":13,"redis_duration_s":0.0017920000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000646,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001146,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010026,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014577,"completed_at":"2026-10-04T19:08:04.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.623391] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc9a5500affad68f5551ef3","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc9a5500affad68f5551ef3: start","job_status":"start","queue_duration_s":1.416626,"scheduling_latency_s":1.416642} gitlab # [ 251.636289] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dcc9a5500affad68f5551ef3","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dcc9a5500affad68f5551ef3: done: 0.016204 sec","job_status":"done","queue_duration_s":1.416626,"scheduling_latency_s":1.416642,"redis_calls":13,"redis_duration_s":0.001979,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001279,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016204,"completed_at":"2026-10-04T19:08:04.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.676729] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f9316cc4d05785c4e56a52e","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f9316cc4d05785c4e56a52e: start","job_status":"start","queue_duration_s":1.434008,"scheduling_latency_s":1.434024} gitlab # [ 251.689455] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9f9316cc4d05785c4e56a52e","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9f9316cc4d05785c4e56a52e: done: 0.015273 sec","job_status":"done","queue_duration_s":1.434008,"scheduling_latency_s":1.434024,"redis_calls":13,"redis_duration_s":0.001958,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001269,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015273,"completed_at":"2026-10-04T19:08:04.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.729840] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbf2348a78ef5a0e971c3b4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbf2348a78ef5a0e971c3b4: start","job_status":"start","queue_duration_s":1.450554,"scheduling_latency_s":1.45058} gitlab # [ 251.742876] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fbf2348a78ef5a0e971c3b4","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fbf2348a78ef5a0e971c3b4: done: 0.015186 sec","job_status":"done","queue_duration_s":1.450554,"scheduling_latency_s":1.45058,"redis_calls":13,"redis_duration_s":0.001875,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000642,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001233,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015186,"completed_at":"2026-10-04T19:08:04.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 251.783916] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc865d90615095946387f4e","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc865d90615095946387f4e: start","job_status":"start","queue_duration_s":1.467582,"scheduling_latency_s":1.467598} gitlab # [ 251.798196] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:08:04.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9cc865d90615095946387f4e","created_at":"2026-10-04T19:08:02.728Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-6477891af6974b4c","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"830b1e917a0dcade9246e7d9a8486900","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:08:02.803Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9cc865d90615095946387f4e: done: 0.016407 sec","job_status":"done","queue_duration_s":1.467582,"scheduling_latency_s":1.467598,"redis_calls":13,"redis_duration_s":0.002075,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000777,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016407,"completed_at":"2026-10-04T19:08:04.287Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 274.959422] setup-gitlab-start[3023]: W, [2026-10-04T19:08:29.880768 #3023] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 306.528431] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.448Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1e6d4b30d52e373ae11d72a8","created_at":"2026-10-04T19:09:01.408Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a46585df3f334c4e08abe9770adbdcd","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.445Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1e6d4b30d52e373ae11d72a8: start","job_status":"start","queue_duration_s":0.00282,"scheduling_latency_s":0.002865} gitlab # [ 306.653241] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.574Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1e6d4b30d52e373ae11d72a8","created_at":"2026-10-04T19:09:01.408Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a46585df3f334c4e08abe9770adbdcd","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.445Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1e6d4b30d52e373ae11d72a8: done: 0.12623 sec","job_status":"done","queue_duration_s":0.00282,"scheduling_latency_s":0.002865,"redis_calls":18,"redis_duration_s":0.026809,"redis_read_bytes":1035,"redis_write_bytes":1637,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.016753,"redis_feature_flag_read_bytes":1028,"redis_feature_flag_write_bytes":550,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006393,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003663,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00146564,"exclusive_lock_hold_duration_s":0.01256854,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.12623,"completed_at":"2026-10-04T19:09:01.574Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 306.713578] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.618Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"42f28c71339e8e68a64a3e85","created_at":"2026-10-04T19:09:01.477Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f06c1a376dbbd98ee5485a4bdb616232","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:09:01.481Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-42f28c71339e8e68a64a3e85: start","job_status":"start","queue_duration_s":0.137002,"scheduling_latency_s":0.137035} gitlab # [ 306.727965] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.642Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 306.737577] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.659Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"42f28c71339e8e68a64a3e85","created_at":"2026-10-04T19:09:01.477Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f06c1a376dbbd98ee5485a4bdb616232","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:09:01.481Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-42f28c71339e8e68a64a3e85: done: 0.040451 sec","job_status":"done","queue_duration_s":0.137002,"scheduling_latency_s":0.137035,"redis_calls":8,"redis_duration_s":0.011324,"redis_read_bytes":133,"redis_write_bytes":908,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.001136,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":107,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008314,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001874,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040451,"completed_at":"2026-10-04T19:09:01.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 306.784852] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91b4abb6c51d657ac8352700","created_at":"2026-10-04T19:09:01.492Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9087aa3a086eaedf06ababdb7718fe33","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:09:01.496Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91b4abb6c51d657ac8352700: start","job_status":"start","queue_duration_s":0.165385,"scheduling_latency_s":0.165416} gitlab # [ 306.866167] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.786Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"91b4abb6c51d657ac8352700","created_at":"2026-10-04T19:09:01.492Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9087aa3a086eaedf06ababdb7718fe33","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:09:01.496Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-91b4abb6c51d657ac8352700: done: 0.123698 sec","job_status":"done","queue_duration_s":0.165385,"scheduling_latency_s":0.165416,"redis_calls":9,"redis_duration_s":0.022966,"redis_read_bytes":261,"redis_write_bytes":906,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002073,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":165,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012114,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.008779,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.123698,"completed_at":"2026-10-04T19:09:01.785Z","load_balancing_strategy":"primary","db_duration_s":0.001363,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 306.925714] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0523997a894b999a08a1a12d","created_at":"2026-10-04T19:09:01.517Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eca9f4ccc8f0d616af724eb8aedc5cea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.528Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0523997a894b999a08a1a12d: start","job_status":"start","queue_duration_s":0.312332,"scheduling_latency_s":0.312367} gitlab # [ 307.042494] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.963Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"0523997a894b999a08a1a12d","created_at":"2026-10-04T19:09:01.517Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eca9f4ccc8f0d616af724eb8aedc5cea","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.528Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-0523997a894b999a08a1a12d: done: 0.122497 sec","job_status":"done","queue_duration_s":0.312332,"scheduling_latency_s":0.312367,"redis_calls":11,"redis_duration_s":0.00969,"redis_read_bytes":263,"redis_write_bytes":1334,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004518,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":172,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002398,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002774,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034593,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00218706,"exclusive_lock_hold_duration_s":0.03325052,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.122497,"completed_at":"2026-10-04T19:09:01.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018991,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.109004] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:01.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dbc1e59f760a7991d307ba87","created_at":"2026-10-04T19:09:01.544Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d535022b641394da09538b0230749ec4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.553Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dbc1e59f760a7991d307ba87: start","job_status":"start","queue_duration_s":0.412597,"scheduling_latency_s":0.412621} gitlab # [ 307.290362] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"dbc1e59f760a7991d307ba87","created_at":"2026-10-04T19:09:01.544Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d535022b641394da09538b0230749ec4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.553Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-dbc1e59f760a7991d307ba87: done: 0.246281 sec","job_status":"done","queue_duration_s":0.412597,"scheduling_latency_s":0.412621,"redis_calls":10,"redis_duration_s":0.00212,"redis_read_bytes":135,"redis_write_bytes":1071,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000198,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":84,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001005,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.205518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0006564,"exclusive_lock_hold_duration_s":0.22522846,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.246281,"completed_at":"2026-10-04T19:09:02.211Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051493,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.360366] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.213Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8f4b38b8731039f9db46193b","created_at":"2026-10-04T19:09:01.566Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.570Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8f4b38b8731039f9db46193b: start","job_status":"start","queue_duration_s":0.642819,"scheduling_latency_s":0.64284} gitlab # [ 307.379230] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8f4b38b8731039f9db46193b","created_at":"2026-10-04T19:09:01.566Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.570Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8f4b38b8731039f9db46193b: done: 0.032846 sec","job_status":"done","queue_duration_s":0.642819,"scheduling_latency_s":0.64284,"redis_calls":16,"redis_duration_s":0.003027,"redis_read_bytes":272,"redis_write_bytes":4225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000333,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":199,"redis_queues_calls":2,"redis_queues_duration_s":0.000292,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001827,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032846,"completed_at":"2026-10-04T19:09:02.246Z","load_balancing_strategy":"primary","db_duration_s":0.001395,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.426421] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9616ce99a72fcdfb552d3d38","created_at":"2026-10-04T19:09:01.585Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6ea158b8fb1b1ab7b1b055df063e431","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.589Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9616ce99a72fcdfb552d3d38: start","job_status":"start","queue_duration_s":0.659281,"scheduling_latency_s":0.659299} gitlab # [ 307.441479] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.257Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 307.444683] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9616ce99a72fcdfb552d3d38","created_at":"2026-10-04T19:09:01.585Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f6ea158b8fb1b1ab7b1b055df063e431","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.589Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9616ce99a72fcdfb552d3d38: done: 0.014794 sec","job_status":"done","queue_duration_s":0.659281,"scheduling_latency_s":0.659299,"redis_calls":8,"redis_duration_s":0.001341,"redis_read_bytes":133,"redis_write_bytes":873,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000152,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":102,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000864,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014794,"completed_at":"2026-10-04T19:09:02.263Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.488343] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"063d24cfe091bd1f90fd312c","created_at":"2026-10-04T19:09:01.615Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1410da363abcf341402e0121f4b23347","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.622Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-063d24cfe091bd1f90fd312c: start","job_status":"start","queue_duration_s":0.642178,"scheduling_latency_s":0.642196} gitlab # [ 307.502890] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.281Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"063d24cfe091bd1f90fd312c","created_at":"2026-10-04T19:09:01.615Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1410da363abcf341402e0121f4b23347","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.622Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-063d24cfe091bd1f90fd312c: done: 0.016968 sec","job_status":"done","queue_duration_s":0.642178,"scheduling_latency_s":0.642196,"redis_calls":8,"redis_duration_s":0.00138,"redis_read_bytes":133,"redis_write_bytes":764,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000151,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016968,"completed_at":"2026-10-04T19:09:02.281Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000963,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.547588] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b1bd92d6639cf1366811f2ac","created_at":"2026-10-04T19:09:01.641Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f159afad5906bb1df3c70da28b036d43","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.653Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b1bd92d6639cf1366811f2ac: start","job_status":"start","queue_duration_s":0.629728,"scheduling_latency_s":0.629747} gitlab # [ 307.562784] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.305Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"b1bd92d6639cf1366811f2ac","created_at":"2026-10-04T19:09:01.641Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f159afad5906bb1df3c70da28b036d43","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.653Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-b1bd92d6639cf1366811f2ac: done: 0.022617 sec","job_status":"done","queue_duration_s":0.629728,"scheduling_latency_s":0.629747,"redis_calls":11,"redis_duration_s":0.002201,"redis_read_bytes":136,"redis_write_bytes":1339,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.00015,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":100,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00113,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000921,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015341,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00504198,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.022617,"completed_at":"2026-10-04T19:09:02.305Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002271,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 307.608714] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.307Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eedf5ef6c39c5a93d6a06f29","created_at":"2026-10-04T19:09:01.688Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e50cc21bd47c3cfccbaffa09dc8dce67","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.699Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eedf5ef6c39c5a93d6a06f29: start","job_status":"start","queue_duration_s":0.608104,"scheduling_latency_s":0.60812} gitlab # [ 307.623322] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"eedf5ef6c39c5a93d6a06f29","created_at":"2026-10-04T19:09:01.688Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e50cc21bd47c3cfccbaffa09dc8dce67","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.699Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-eedf5ef6c39c5a93d6a06f29: done: 0.0276 sec","job_status":"done","queue_duration_s":0.608104,"scheduling_latency_s":0.60812,"redis_calls":11,"redis_duration_s":0.0018930000000000002,"redis_read_bytes":517,"redis_write_bytes":1011,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000633,"redis_feature_flag_read_bytes":512,"redis_feature_flag_write_bytes":300,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000938,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000322,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014408,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0276,"completed_at":"2026-10-04T19:09:02.335Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008435,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.666982] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"670a07691abf3a953db46f29","created_at":"2026-10-04T19:09:01.702Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.712Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-670a07691abf3a953db46f29: start","job_status":"start","queue_duration_s":0.624028,"scheduling_latency_s":0.624047} gitlab # [ 307.681152] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"670a07691abf3a953db46f29","created_at":"2026-10-04T19:09:01.702Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.712Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-670a07691abf3a953db46f29: done: 0.094654 sec","job_status":"done","queue_duration_s":0.624028,"scheduling_latency_s":0.624047,"redis_calls":65,"redis_duration_s":0.011909,"redis_read_bytes":748,"redis_write_bytes":41502,"redis_feature_flag_calls":5,"redis_feature_flag_duration_s":0.00078,"redis_feature_flag_read_bytes":640,"redis_feature_flag_write_bytes":393,"redis_queues_calls":2,"redis_queues_duration_s":0.000722,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009892,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000515,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085144,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.094654,"completed_at":"2026-10-04T19:09:02.430Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.724985] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.432Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bd95769d0c48b6be9b30411c","created_at":"2026-10-04T19:09:01.717Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"360b598a5bd3279d9e248ef02e6d95b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.737Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bd95769d0c48b6be9b30411c: start","job_status":"start","queue_duration_s":0.694923,"scheduling_latency_s":0.694939} gitlab # [ 307.740000] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.446Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"bd95769d0c48b6be9b30411c","created_at":"2026-10-04T19:09:01.717Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"360b598a5bd3279d9e248ef02e6d95b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.737Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-bd95769d0c48b6be9b30411c: done: 0.014191 sec","job_status":"done","queue_duration_s":0.694923,"scheduling_latency_s":0.694939,"redis_calls":9,"redis_duration_s":0.00147,"redis_read_bytes":261,"redis_write_bytes":944,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000294,"redis_feature_flag_read_bytes":256,"redis_feature_flag_write_bytes":168,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000846,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014191,"completed_at":"2026-10-04T19:09:02.446Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.784043] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.447Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"70b00a581a1425a58a92a818","created_at":"2026-10-04T19:09:01.781Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e46f13a72bf200f0884afd1294bc6b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.787Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-70b00a581a1425a58a92a818: start","job_status":"start","queue_duration_s":0.659665,"scheduling_latency_s":0.65968} gitlab # [ 307.798962] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"70b00a581a1425a58a92a818","created_at":"2026-10-04T19:09:01.781Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e46f13a72bf200f0884afd1294bc6b7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.787Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-70b00a581a1425a58a92a818: done: 0.013961 sec","job_status":"done","queue_duration_s":0.659665,"scheduling_latency_s":0.65968,"redis_calls":8,"redis_duration_s":0.001338,"redis_read_bytes":133,"redis_write_bytes":943,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000151,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":112,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000868,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013961,"completed_at":"2026-10-04T19:09:02.461Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.843117] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.462Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0817e25e1c23287080fc81ab","created_at":"2026-10-04T19:09:01.800Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a662acc75889f4607cdf1c52fe19281","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.805Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0817e25e1c23287080fc81ab: start","job_status":"start","queue_duration_s":0.657727,"scheduling_latency_s":0.657743} gitlab # [ 307.858481] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.502Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0817e25e1c23287080fc81ab","created_at":"2026-10-04T19:09:01.800Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a662acc75889f4607cdf1c52fe19281","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.805Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0817e25e1c23287080fc81ab: done: 0.039627 sec","job_status":"done","queue_duration_s":0.657727,"scheduling_latency_s":0.657743,"redis_calls":10,"redis_duration_s":0.0019270000000000001,"redis_read_bytes":135,"redis_write_bytes":1275,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000167,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":106,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000921,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000839,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057148,"exclusive_lock_hold_duration_s":0.0194274,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.039627,"completed_at":"2026-10-04T19:09:02.502Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012576,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.905021] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f404b9e322c278c1a60cd529","created_at":"2026-10-04T19:09:01.839Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9db0d3f7733d55885aa7a1b440f7393","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.846Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f404b9e322c278c1a60cd529: start","job_status":"start","queue_duration_s":0.656995,"scheduling_latency_s":0.657011} gitlab # [ 307.919052] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"f404b9e322c278c1a60cd529","created_at":"2026-10-04T19:09:01.839Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c9db0d3f7733d55885aa7a1b440f7393","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.846Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-f404b9e322c278c1a60cd529: done: 0.017298 sec","job_status":"done","queue_duration_s":0.656995,"scheduling_latency_s":0.657011,"redis_calls":10,"redis_duration_s":0.001841,"redis_read_bytes":135,"redis_write_bytes":1167,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000172,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":95,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00050846,"exclusive_lock_hold_duration_s":0.0032911,"duration_s":0.017298,"completed_at":"2026-10-04T19:09:02.520Z","load_balancing_strategy":"primary","db_duration_s":0.001026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 307.963157] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.522Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b41a666fae680278e3d586d5","created_at":"2026-10-04T19:09:01.851Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9707b877d6745a5731f18ece786ab7cc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.854Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b41a666fae680278e3d586d5: start","job_status":"start","queue_duration_s":0.668291,"scheduling_latency_s":0.668308} gitlab # [ 307.977749] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.547Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b41a666fae680278e3d586d5","created_at":"2026-10-04T19:09:01.851Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9707b877d6745a5731f18ece786ab7cc","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.854Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b41a666fae680278e3d586d5: done: 0.024905 sec","job_status":"done","queue_duration_s":0.668291,"scheduling_latency_s":0.668308,"redis_calls":8,"redis_duration_s":0.0021290000000000002,"redis_read_bytes":133,"redis_write_bytes":936,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000149,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000506,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.024905,"completed_at":"2026-10-04T19:09:02.547Z","load_balancing_strategy":"primary","db_duration_s":0.00702,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.021927] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ccb2b82b6745846ac0805841","created_at":"2026-10-04T19:09:01.859Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79a957ff98876abc93625ba25fdba7a8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.862Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ccb2b82b6745846ac0805841: start","job_status":"start","queue_duration_s":0.686433,"scheduling_latency_s":0.686457} gitlab # [ 308.036521] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"ccb2b82b6745846ac0805841","created_at":"2026-10-04T19:09:01.859Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"79a957ff98876abc93625ba25fdba7a8","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.862Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-ccb2b82b6745846ac0805841: done: 0.365615 sec","job_status":"done","queue_duration_s":0.686433,"scheduling_latency_s":0.686457,"redis_calls":563,"redis_duration_s":0.072458,"redis_read_bytes":688,"redis_write_bytes":48843,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000168,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":83,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000966,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.071324,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.33524,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.365615,"completed_at":"2026-10-04T19:09:02.914Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.079814] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.916Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5a6e999c1823de61a18c8be5","created_at":"2026-10-04T19:09:01.881Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c539e582477e23a0a0ef07d363cd37c9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.887Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5a6e999c1823de61a18c8be5: start","job_status":"start","queue_duration_s":1.029073,"scheduling_latency_s":1.0291} gitlab # [ 308.095065] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.925Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 308.098724] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"5a6e999c1823de61a18c8be5","created_at":"2026-10-04T19:09:01.881Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c539e582477e23a0a0ef07d363cd37c9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.887Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-5a6e999c1823de61a18c8be5: done: 0.014305 sec","job_status":"done","queue_duration_s":1.029073,"scheduling_latency_s":1.0291,"redis_calls":8,"redis_duration_s":0.00124,"redis_read_bytes":133,"redis_write_bytes":937,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000137,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":111,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000783,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014305,"completed_at":"2026-10-04T19:09:02.930Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.142479] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.932Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d9ee9004bb0b4327c94e6359","created_at":"2026-10-04T19:09:01.893Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3430561d42c5d6066c65ffaef2ca5a79","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.897Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d9ee9004bb0b4327c94e6359: start","job_status":"start","queue_duration_s":1.034217,"scheduling_latency_s":1.034242} gitlab # [ 308.157655] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.941Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 308.162358] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.947Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d9ee9004bb0b4327c94e6359","created_at":"2026-10-04T19:09:01.893Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3430561d42c5d6066c65ffaef2ca5a79","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.897Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d9ee9004bb0b4327c94e6359: done: 0.015231 sec","job_status":"done","queue_duration_s":1.034217,"scheduling_latency_s":1.034242,"redis_calls":8,"redis_duration_s":0.0023539999999999998,"redis_read_bytes":133,"redis_write_bytes":931,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000171,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":110,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001849,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015231,"completed_at":"2026-10-04T19:09:02.947Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.207037] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.948Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4b1d48d40b5026b0f090173b","created_at":"2026-10-04T19:09:01.914Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fb9b83fd720755642d26c9eceb235c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.920Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4b1d48d40b5026b0f090173b: start","job_status":"start","queue_duration_s":1.028223,"scheduling_latency_s":1.028249} gitlab # [ 308.222187] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.957Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 308.225942] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"4b1d48d40b5026b0f090173b","created_at":"2026-10-04T19:09:01.914Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9fb9b83fd720755642d26c9eceb235c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:01.920Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-4b1d48d40b5026b0f090173b: done: 0.013922 sec","job_status":"done","queue_duration_s":1.028223,"scheduling_latency_s":1.028249,"redis_calls":8,"redis_duration_s":0.001356,"redis_read_bytes":133,"redis_write_bytes":867,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000152,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":101,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000344,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013922,"completed_at":"2026-10-04T19:09:02.962Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.269893] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.964Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"82970d0ce24e8a6113542756","created_at":"2026-10-04T19:09:01.928Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0c2d1a515cd3103486c8584c3fb61ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:09:01.934Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-82970d0ce24e8a6113542756: start","job_status":"start","queue_duration_s":1.029512,"scheduling_latency_s":1.029537} gitlab # [ 308.284130] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.972Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 308.287926] redis[794]: 10 changes in 300 seconds. Saving... gitlab # [ 308.289046] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.978Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"82970d0ce24e8a6113542756","created_at":"2026-10-04T19:09:01.928Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-37325aae2cc14b68","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e0c2d1a515cd3103486c8584c3fb61ed","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:09:01.934Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-82970d0ce24e8a6113542756: done: 0.014052 sec","job_status":"done","queue_duration_s":1.029512,"scheduling_latency_s":1.029537,"redis_calls":8,"redis_duration_s":0.001454,"redis_read_bytes":133,"redis_write_bytes":914,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000164,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":108,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000368,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014052,"completed_at":"2026-10-04T19:09:02.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.331512] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.979Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13cd9afa607a6bb9d34d73e0","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13cd9afa607a6bb9d34d73e0: start","job_status":"start","queue_duration_s":0.739421,"scheduling_latency_s":0.739442} gitlab # [ 308.344367] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:02.994Z","message":"No existing merge request to be cleaned up."} gitlab # [ 308.346051] redis[794]: Background saving started by pid 3351 gitlab # [ 308.347004] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.003Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"13cd9afa607a6bb9d34d73e0","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-13cd9afa607a6bb9d34d73e0: done: 0.024352 sec","job_status":"done","queue_duration_s":0.739421,"scheduling_latency_s":0.739442,"redis_calls":11,"redis_duration_s":0.004685,"redis_read_bytes":136,"redis_write_bytes":1104,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002846,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":82,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001143,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01545,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024352,"completed_at":"2026-10-04T19:09:03.003Z","load_balancing_strategy":"primary","db_duration_s":0.002111,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.389539] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.005Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a141b996748de329602cee57","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a141b996748de329602cee57: start","job_status":"start","queue_duration_s":0.765109,"scheduling_latency_s":0.765131} gitlab # [ 308.402229] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.015Z","message":"No existing merge request to be cleaned up."} gitlab # [ 308.404093] redis[3351]: BGSAVE done, 503 keys saved, 0 keys skipped, 127109 bytes written. gitlab # [ 308.405403] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.024Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"a141b996748de329602cee57","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-a141b996748de329602cee57: done: 0.01909 sec","job_status":"done","queue_duration_s":0.765109,"scheduling_latency_s":0.765131,"redis_calls":10,"redis_duration_s":0.001827,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001047,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01909,"completed_at":"2026-10-04T19:09:03.024Z","load_balancing_strategy":"primary","db_duration_s":0.001577,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.445483] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.025Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcca7843bbcedb36eef68672","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcca7843bbcedb36eef68672: start","job_status":"start","queue_duration_s":0.785512,"scheduling_latency_s":0.785532} gitlab # [ 308.458467] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.035Z","message":"No existing merge request to be cleaned up."} gitlab # [ 308.460201] redis[3351]: DB saved on disk gitlab # [ 308.460951] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.043Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"fcca7843bbcedb36eef68672","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-fcca7843bbcedb36eef68672: done: 0.018005 sec","job_status":"done","queue_duration_s":0.785512,"scheduling_latency_s":0.785532,"redis_calls":10,"redis_duration_s":0.001716,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000731,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012307,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018005,"completed_at":"2026-10-04T19:09:03.043Z","load_balancing_strategy":"primary","db_duration_s":0.001758,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.500760] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.044Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d518f7e69cac9acb0b14a0a3","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d518f7e69cac9acb0b14a0a3: start","job_status":"start","queue_duration_s":0.804686,"scheduling_latency_s":0.804706} gitlab # [ 308.513476] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.054Z","message":"No existing merge request to be cleaned up."} gitlab # [ 308.515156] redis[3351]: Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB gitlab # [ 308.516356] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.062Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d518f7e69cac9acb0b14a0a3","created_at":"2026-10-04T19:09:02.227Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-099b714a85624210","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"407d30313fef2a8aa4184551d1d4c2db","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.240Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d518f7e69cac9acb0b14a0a3: done: 0.017718 sec","job_status":"done","queue_duration_s":0.804686,"scheduling_latency_s":0.804706,"redis_calls":10,"redis_duration_s":0.001697,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000707,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017718,"completed_at":"2026-10-04T19:09:03.062Z","load_balancing_strategy":"primary","db_duration_s":0.001505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.556844] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d4d361ed31e4fed3bff9ca","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d4d361ed31e4fed3bff9ca: start","job_status":"start","queue_duration_s":0.639716,"scheduling_latency_s":0.639737} gitlab # [ 308.570649] redis[794]: Background saving terminated with success gitlab # [ 308.571693] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05d4d361ed31e4fed3bff9ca","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05d4d361ed31e4fed3bff9ca: done: 0.015861 sec","job_status":"done","queue_duration_s":0.639716,"scheduling_latency_s":0.639737,"redis_calls":14,"redis_duration_s":0.002329,"redis_read_bytes":138,"redis_write_bytes":1242,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000177,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":85,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001448,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010531,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015861,"completed_at":"2026-10-04T19:09:03.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.614452] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e26736d0f1bc9d8d4c5bf223","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e26736d0f1bc9d8d4c5bf223: start","job_status":"start","queue_duration_s":0.656734,"scheduling_latency_s":0.656754} gitlab # [ 308.627390] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e26736d0f1bc9d8d4c5bf223","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e26736d0f1bc9d8d4c5bf223: done: 0.01754 sec","job_status":"done","queue_duration_s":0.656734,"scheduling_latency_s":0.656754,"redis_calls":13,"redis_duration_s":0.003595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002381,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01754,"completed_at":"2026-10-04T19:09:03.098Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.668132] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.100Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588c91ec151a9ea8e5166cc3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588c91ec151a9ea8e5166cc3: start","job_status":"start","queue_duration_s":0.676422,"scheduling_latency_s":0.676452} gitlab # [ 308.680908] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.123Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"588c91ec151a9ea8e5166cc3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-588c91ec151a9ea8e5166cc3: done: 0.022694 sec","job_status":"done","queue_duration_s":0.676422,"scheduling_latency_s":0.676452,"redis_calls":13,"redis_duration_s":0.002542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000898,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001644,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022694,"completed_at":"2026-10-04T19:09:03.123Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.721409] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.124Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2b29f0b7d1d72a45f3a4e47","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2b29f0b7d1d72a45f3a4e47: start","job_status":"start","queue_duration_s":0.700831,"scheduling_latency_s":0.70086} gitlab # [ 308.734293] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.163Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a2b29f0b7d1d72a45f3a4e47","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a2b29f0b7d1d72a45f3a4e47: done: 0.038764 sec","job_status":"done","queue_duration_s":0.700831,"scheduling_latency_s":0.70086,"redis_calls":13,"redis_duration_s":0.001988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000787,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001201,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038764,"completed_at":"2026-10-04T19:09:03.163Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.775881] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.164Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e999c5dc89241d61b1135196","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e999c5dc89241d61b1135196: start","job_status":"start","queue_duration_s":0.74087,"scheduling_latency_s":0.74089} gitlab # [ 308.789036] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.180Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e999c5dc89241d61b1135196","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e999c5dc89241d61b1135196: done: 0.01593 sec","job_status":"done","queue_duration_s":0.74087,"scheduling_latency_s":0.74089,"redis_calls":13,"redis_duration_s":0.001655,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001059,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01593,"completed_at":"2026-10-04T19:09:03.180Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.829221] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c31a30e003a6e8cec09151c6","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c31a30e003a6e8cec09151c6: start","job_status":"start","queue_duration_s":0.758112,"scheduling_latency_s":0.75813} gitlab # [ 308.842239] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c31a30e003a6e8cec09151c6","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c31a30e003a6e8cec09151c6: done: 0.016635 sec","job_status":"done","queue_duration_s":0.758112,"scheduling_latency_s":0.75813,"redis_calls":13,"redis_duration_s":0.002101,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000614,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001487,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016635,"completed_at":"2026-10-04T19:09:03.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.882885] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b442fdda45581cbf7bea04","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b442fdda45581cbf7bea04: start","job_status":"start","queue_duration_s":0.775886,"scheduling_latency_s":0.775903} gitlab # [ 308.895777] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"48b442fdda45581cbf7bea04","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-48b442fdda45581cbf7bea04: done: 0.015252 sec","job_status":"done","queue_duration_s":0.775886,"scheduling_latency_s":0.775903,"redis_calls":13,"redis_duration_s":0.001677,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000589,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010664,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015252,"completed_at":"2026-10-04T19:09:03.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.936773] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93121fe2f5d8a335e0ce2e4b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93121fe2f5d8a335e0ce2e4b: start","job_status":"start","queue_duration_s":0.792298,"scheduling_latency_s":0.792316} gitlab # [ 308.949549] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"93121fe2f5d8a335e0ce2e4b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-93121fe2f5d8a335e0ce2e4b: done: 0.014511 sec","job_status":"done","queue_duration_s":0.792298,"scheduling_latency_s":0.792316,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014511,"completed_at":"2026-10-04T19:09:03.230Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 308.991107] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911ad74ddf3aefb918badd03","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911ad74ddf3aefb918badd03: start","job_status":"start","queue_duration_s":0.807934,"scheduling_latency_s":0.807951} gitlab # [ 309.004032] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"911ad74ddf3aefb918badd03","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-911ad74ddf3aefb918badd03: done: 0.014547 sec","job_status":"done","queue_duration_s":0.807934,"scheduling_latency_s":0.807951,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014547,"completed_at":"2026-10-04T19:09:03.246Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.044788] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7f6caaba2c73cd7d8d3654e","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7f6caaba2c73cd7d8d3654e: start","job_status":"start","queue_duration_s":0.823605,"scheduling_latency_s":0.823623} gitlab # [ 309.057551] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.262Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f7f6caaba2c73cd7d8d3654e","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f7f6caaba2c73cd7d8d3654e: done: 0.01448 sec","job_status":"done","queue_duration_s":0.823605,"scheduling_latency_s":0.823623,"redis_calls":13,"redis_duration_s":0.001483,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01448,"completed_at":"2026-10-04T19:09:03.262Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.098306] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74704a5499fa2cf1a755dbc7","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74704a5499fa2cf1a755dbc7: start","job_status":"start","queue_duration_s":0.83926,"scheduling_latency_s":0.839277} gitlab # [ 309.111215] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.277Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"74704a5499fa2cf1a755dbc7","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-74704a5499fa2cf1a755dbc7: done: 0.014562 sec","job_status":"done","queue_duration_s":0.83926,"scheduling_latency_s":0.839277,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-10-04T19:09:03.277Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.152561] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.279Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6131124963fb79e8bf0940f3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6131124963fb79e8bf0940f3: start","job_status":"start","queue_duration_s":0.854953,"scheduling_latency_s":0.854971} gitlab # [ 309.165192] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.293Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6131124963fb79e8bf0940f3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6131124963fb79e8bf0940f3: done: 0.014455 sec","job_status":"done","queue_duration_s":0.854953,"scheduling_latency_s":0.854971,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014455,"completed_at":"2026-10-04T19:09:03.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.206818] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.294Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf68628f70599a888bccc444","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf68628f70599a888bccc444: start","job_status":"start","queue_duration_s":0.870571,"scheduling_latency_s":0.87059} gitlab # [ 309.219808] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf68628f70599a888bccc444","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf68628f70599a888bccc444: done: 0.01491 sec","job_status":"done","queue_duration_s":0.870571,"scheduling_latency_s":0.87059,"redis_calls":13,"redis_duration_s":0.00156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010183,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01491,"completed_at":"2026-10-04T19:09:03.309Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.260647] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.310Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d909faf4531675888322fd","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d909faf4531675888322fd: start","job_status":"start","queue_duration_s":0.886625,"scheduling_latency_s":0.886644} gitlab # [ 309.273409] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"64d909faf4531675888322fd","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-64d909faf4531675888322fd: done: 0.014588 sec","job_status":"done","queue_duration_s":0.886625,"scheduling_latency_s":0.886644,"redis_calls":13,"redis_duration_s":0.0015409999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010254,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014588,"completed_at":"2026-10-04T19:09:03.325Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.314286] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32724ae6d28542d632b8b3fa","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32724ae6d28542d632b8b3fa: start","job_status":"start","queue_duration_s":0.902418,"scheduling_latency_s":0.902436} gitlab # [ 309.327421] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.341Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"32724ae6d28542d632b8b3fa","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-32724ae6d28542d632b8b3fa: done: 0.014572 sec","job_status":"done","queue_duration_s":0.902418,"scheduling_latency_s":0.902436,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014572,"completed_at":"2026-10-04T19:09:03.340Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.368237] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.342Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b6cc54ede2693ab2581896c","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b6cc54ede2693ab2581896c: start","job_status":"start","queue_duration_s":0.918103,"scheduling_latency_s":0.918122} gitlab # [ 309.381075] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.356Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b6cc54ede2693ab2581896c","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b6cc54ede2693ab2581896c: done: 0.01453 sec","job_status":"done","queue_duration_s":0.918103,"scheduling_latency_s":0.918122,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01453,"completed_at":"2026-10-04T19:09:03.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.422547] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.358Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"effe1c42408e92ba160a652c","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-effe1c42408e92ba160a652c: start","job_status":"start","queue_duration_s":0.933945,"scheduling_latency_s":0.933965} gitlab # [ 309.435689] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.372Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"effe1c42408e92ba160a652c","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-effe1c42408e92ba160a652c: done: 0.01489 sec","job_status":"done","queue_duration_s":0.933945,"scheduling_latency_s":0.933965,"redis_calls":13,"redis_duration_s":0.001575,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01033,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01489,"completed_at":"2026-10-04T19:09:03.372Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.476602] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df69575dc9056ea75ecdcc75","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df69575dc9056ea75ecdcc75: start","job_status":"start","queue_duration_s":0.949936,"scheduling_latency_s":0.949954} gitlab # [ 309.489246] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.388Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df69575dc9056ea75ecdcc75","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df69575dc9056ea75ecdcc75: done: 0.01437 sec","job_status":"done","queue_duration_s":0.949936,"scheduling_latency_s":0.949954,"redis_calls":13,"redis_duration_s":0.001484,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01437,"completed_at":"2026-10-04T19:09:03.388Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.529803] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d8a02b13bc84c2256fee4c4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d8a02b13bc84c2256fee4c4: start","job_status":"start","queue_duration_s":0.965398,"scheduling_latency_s":0.965415} gitlab # [ 309.542564] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8d8a02b13bc84c2256fee4c4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8d8a02b13bc84c2256fee4c4: done: 0.01478 sec","job_status":"done","queue_duration_s":0.965398,"scheduling_latency_s":0.965415,"redis_calls":13,"redis_duration_s":0.001735,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001192,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01478,"completed_at":"2026-10-04T19:09:03.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.583508] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.405Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef8f50643a3245ceb32db614","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef8f50643a3245ceb32db614: start","job_status":"start","queue_duration_s":0.98128,"scheduling_latency_s":0.981298} gitlab # [ 309.597432] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef8f50643a3245ceb32db614","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef8f50643a3245ceb32db614: done: 0.014482 sec","job_status":"done","queue_duration_s":0.98128,"scheduling_latency_s":0.981298,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000535,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010048,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014482,"completed_at":"2026-10-04T19:09:03.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.638042] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.420Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66851d6158cc5ecbeb2dffd8","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66851d6158cc5ecbeb2dffd8: start","job_status":"start","queue_duration_s":0.996913,"scheduling_latency_s":0.996931} gitlab # [ 309.650973] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66851d6158cc5ecbeb2dffd8","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66851d6158cc5ecbeb2dffd8: done: 0.014534 sec","job_status":"done","queue_duration_s":0.996913,"scheduling_latency_s":0.996931,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014534,"completed_at":"2026-10-04T19:09:03.435Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.691823] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bbe4f2d7fe97ca31debf2cb","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bbe4f2d7fe97ca31debf2cb: start","job_status":"start","queue_duration_s":1.01255,"scheduling_latency_s":1.012569} gitlab # [ 309.704827] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.450Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9bbe4f2d7fe97ca31debf2cb","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9bbe4f2d7fe97ca31debf2cb: done: 0.014374 sec","job_status":"done","queue_duration_s":1.01255,"scheduling_latency_s":1.012569,"redis_calls":13,"redis_duration_s":0.001456,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009952,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014374,"completed_at":"2026-10-04T19:09:03.450Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.745119] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.452Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"065e605be43ace1434b12524","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-065e605be43ace1434b12524: start","job_status":"start","queue_duration_s":1.028014,"scheduling_latency_s":1.028032} gitlab # [ 309.757775] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.466Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"065e605be43ace1434b12524","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-065e605be43ace1434b12524: done: 0.014522 sec","job_status":"done","queue_duration_s":1.028014,"scheduling_latency_s":1.028032,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014522,"completed_at":"2026-10-04T19:09:03.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.800026] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.467Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2abff8b810f83d6627ab4e4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2abff8b810f83d6627ab4e4: start","job_status":"start","queue_duration_s":1.043686,"scheduling_latency_s":1.043704} gitlab # [ 309.812809] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2abff8b810f83d6627ab4e4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2abff8b810f83d6627ab4e4: done: 0.016318 sec","job_status":"done","queue_duration_s":1.043686,"scheduling_latency_s":1.043704,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016318,"completed_at":"2026-10-04T19:09:03.483Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.853077] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9d6be221f783d89af9b33a4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9d6be221f783d89af9b33a4: start","job_status":"start","queue_duration_s":1.061101,"scheduling_latency_s":1.061119} gitlab # [ 309.866116] postgres[864]: [864] LOG: checkpoint starting: time gitlab # [ 309.866990] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.499Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9d6be221f783d89af9b33a4","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9d6be221f783d89af9b33a4: done: 0.014718 sec","job_status":"done","queue_duration_s":1.061101,"scheduling_latency_s":1.061119,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014718,"completed_at":"2026-10-04T19:09:03.499Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.907545] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.500Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0ea440b32938c8e8428d50","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0ea440b32938c8e8428d50: start","job_status":"start","queue_duration_s":1.076909,"scheduling_latency_s":1.076926} gitlab # [ 309.920584] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.515Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e0ea440b32938c8e8428d50","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e0ea440b32938c8e8428d50: done: 0.014474 sec","job_status":"done","queue_duration_s":1.076909,"scheduling_latency_s":1.076926,"redis_calls":13,"redis_duration_s":0.0014650000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01001,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014474,"completed_at":"2026-10-04T19:09:03.515Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 309.961123] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.516Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30fe57ff561c68b1de83b3b3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30fe57ff561c68b1de83b3b3: start","job_status":"start","queue_duration_s":1.092501,"scheduling_latency_s":1.092518} gitlab # [ 309.973971] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.531Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30fe57ff561c68b1de83b3b3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30fe57ff561c68b1de83b3b3: done: 0.014896 sec","job_status":"done","queue_duration_s":1.092501,"scheduling_latency_s":1.092518,"redis_calls":13,"redis_duration_s":0.001634,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001077,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014896,"completed_at":"2026-10-04T19:09:03.531Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.016256] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.532Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fed5b30e324e6f2c3f01efa","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fed5b30e324e6f2c3f01efa: start","job_status":"start","queue_duration_s":1.108495,"scheduling_latency_s":1.108513} gitlab # [ 310.029095] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.547Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fed5b30e324e6f2c3f01efa","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fed5b30e324e6f2c3f01efa: done: 0.014466 sec","job_status":"done","queue_duration_s":1.108495,"scheduling_latency_s":1.108513,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014466,"completed_at":"2026-10-04T19:09:03.546Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.069343] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.548Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6546af8da8f425692cf1689","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6546af8da8f425692cf1689: start","job_status":"start","queue_duration_s":1.124044,"scheduling_latency_s":1.124062} gitlab # [ 310.081985] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6546af8da8f425692cf1689","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6546af8da8f425692cf1689: done: 0.019322 sec","job_status":"done","queue_duration_s":1.124044,"scheduling_latency_s":1.124062,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019322,"completed_at":"2026-10-04T19:09:03.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.122486] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35d2a437f66bcbb92d600e31","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35d2a437f66bcbb92d600e31: start","job_status":"start","queue_duration_s":1.144522,"scheduling_latency_s":1.14454} gitlab # [ 310.135314] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35d2a437f66bcbb92d600e31","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35d2a437f66bcbb92d600e31: done: 0.014604 sec","job_status":"done","queue_duration_s":1.144522,"scheduling_latency_s":1.14454,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010074,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014604,"completed_at":"2026-10-04T19:09:03.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.176460] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6cae8e93c85b9d64526f253","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6cae8e93c85b9d64526f253: start","job_status":"start","queue_duration_s":1.160261,"scheduling_latency_s":1.16028} gitlab # [ 310.189539] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6cae8e93c85b9d64526f253","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6cae8e93c85b9d64526f253: done: 0.014876 sec","job_status":"done","queue_duration_s":1.160261,"scheduling_latency_s":1.16028,"redis_calls":13,"redis_duration_s":0.001498,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000932,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014876,"completed_at":"2026-10-04T19:09:03.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.231076] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df74d0b4faf2f1c931ab23a3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df74d0b4faf2f1c931ab23a3: start","job_status":"start","queue_duration_s":1.176272,"scheduling_latency_s":1.17629} gitlab # [ 310.244120] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"df74d0b4faf2f1c931ab23a3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-df74d0b4faf2f1c931ab23a3: done: 0.014443 sec","job_status":"done","queue_duration_s":1.176272,"scheduling_latency_s":1.17629,"redis_calls":13,"redis_duration_s":0.0014750000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014443,"completed_at":"2026-10-04T19:09:03.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.285005] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a3eb9b10302c1000226a79","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a3eb9b10302c1000226a79: start","job_status":"start","queue_duration_s":1.191834,"scheduling_latency_s":1.191852} gitlab # [ 310.297802] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81a3eb9b10302c1000226a79","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81a3eb9b10302c1000226a79: done: 0.0146 sec","job_status":"done","queue_duration_s":1.191834,"scheduling_latency_s":1.191852,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009996,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0146,"completed_at":"2026-10-04T19:09:03.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.338377] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e07a027e8b1fcf8896567523","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e07a027e8b1fcf8896567523: start","job_status":"start","queue_duration_s":1.207549,"scheduling_latency_s":1.207567} gitlab # [ 310.351221] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e07a027e8b1fcf8896567523","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e07a027e8b1fcf8896567523: done: 0.014333 sec","job_status":"done","queue_duration_s":1.207549,"scheduling_latency_s":1.207567,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009899,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014333,"completed_at":"2026-10-04T19:09:03.645Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.392665] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.647Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee2c5ec9d5455718c777d65b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee2c5ec9d5455718c777d65b: start","job_status":"start","queue_duration_s":1.223002,"scheduling_latency_s":1.22302} gitlab # [ 310.406346] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.661Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee2c5ec9d5455718c777d65b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee2c5ec9d5455718c777d65b: done: 0.014589 sec","job_status":"done","queue_duration_s":1.223002,"scheduling_latency_s":1.22302,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014589,"completed_at":"2026-10-04T19:09:03.661Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.446890] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.662Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d133cc5cc7e3d40fff096f","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d133cc5cc7e3d40fff096f: start","job_status":"start","queue_duration_s":1.238735,"scheduling_latency_s":1.238753} gitlab # [ 310.460022] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"52d133cc5cc7e3d40fff096f","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-52d133cc5cc7e3d40fff096f: done: 0.014403 sec","job_status":"done","queue_duration_s":1.238735,"scheduling_latency_s":1.238753,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000949,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014403,"completed_at":"2026-10-04T19:09:03.677Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.500632] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.678Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81da816fa4c44a02679679d9","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81da816fa4c44a02679679d9: start","job_status":"start","queue_duration_s":1.254296,"scheduling_latency_s":1.254314} gitlab # [ 310.513541] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.692Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"81da816fa4c44a02679679d9","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-81da816fa4c44a02679679d9: done: 0.014307 sec","job_status":"done","queue_duration_s":1.254296,"scheduling_latency_s":1.254314,"redis_calls":13,"redis_duration_s":0.00145,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009937,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014307,"completed_at":"2026-10-04T19:09:03.692Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.553741] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed9ba2c5c86fffc6e005079b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed9ba2c5c86fffc6e005079b: start","job_status":"start","queue_duration_s":1.269729,"scheduling_latency_s":1.269747} gitlab # [ 310.566547] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.708Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed9ba2c5c86fffc6e005079b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed9ba2c5c86fffc6e005079b: done: 0.015065 sec","job_status":"done","queue_duration_s":1.269729,"scheduling_latency_s":1.269747,"redis_calls":13,"redis_duration_s":0.001889,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000757,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015065,"completed_at":"2026-10-04T19:09:03.708Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.607336] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdcad17ce736978b1ec76c93","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdcad17ce736978b1ec76c93: start","job_status":"start","queue_duration_s":1.285952,"scheduling_latency_s":1.285971} gitlab # [ 310.621357] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cdcad17ce736978b1ec76c93","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cdcad17ce736978b1ec76c93: done: 0.014392 sec","job_status":"done","queue_duration_s":1.285952,"scheduling_latency_s":1.285971,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014392,"completed_at":"2026-10-04T19:09:03.724Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.661744] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.725Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78636be9fdcc4d2ac4c127b3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78636be9fdcc4d2ac4c127b3: start","job_status":"start","queue_duration_s":1.301476,"scheduling_latency_s":1.301494} gitlab # [ 310.674542] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.740Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"78636be9fdcc4d2ac4c127b3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-78636be9fdcc4d2ac4c127b3: done: 0.014567 sec","job_status":"done","queue_duration_s":1.301476,"scheduling_latency_s":1.301494,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009988,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014567,"completed_at":"2026-10-04T19:09:03.740Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.715354] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f94d80a009a01c660f1712","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f94d80a009a01c660f1712: start","job_status":"start","queue_duration_s":1.317154,"scheduling_latency_s":1.317172} gitlab # [ 310.728587] gitaly[923]: time="2026-10-04T19:09:05.588Z" level=info msg="burden monitor stats" pid=923 gitlab # [ 310.729868] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.755Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f94d80a009a01c660f1712","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f94d80a009a01c660f1712: done: 0.014668 sec","job_status":"done","queue_duration_s":1.317154,"scheduling_latency_s":1.317172,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014668,"completed_at":"2026-10-04T19:09:03.755Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.770311] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a932f445af0a1af0a7bc2ad","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a932f445af0a1af0a7bc2ad: start","job_status":"start","queue_duration_s":1.332977,"scheduling_latency_s":1.332995} gitlab # [ 310.783244] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.771Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8a932f445af0a1af0a7bc2ad","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8a932f445af0a1af0a7bc2ad: done: 0.014571 sec","job_status":"done","queue_duration_s":1.332977,"scheduling_latency_s":1.332995,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010058,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014571,"completed_at":"2026-10-04T19:09:03.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.824997] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878aa6e345a6593faa1eaac0","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878aa6e345a6593faa1eaac0: start","job_status":"start","queue_duration_s":1.348746,"scheduling_latency_s":1.348764} gitlab # [ 310.837830] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.787Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"878aa6e345a6593faa1eaac0","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-878aa6e345a6593faa1eaac0: done: 0.014366 sec","job_status":"done","queue_duration_s":1.348746,"scheduling_latency_s":1.348764,"redis_calls":13,"redis_duration_s":0.0014689999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00094,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009932,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014366,"completed_at":"2026-10-04T19:09:03.787Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.878205] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.788Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bf23930a15af9cee253803d","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bf23930a15af9cee253803d: start","job_status":"start","queue_duration_s":1.364228,"scheduling_latency_s":1.364246} gitlab # [ 310.891097] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.803Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4bf23930a15af9cee253803d","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4bf23930a15af9cee253803d: done: 0.014955 sec","job_status":"done","queue_duration_s":1.364228,"scheduling_latency_s":1.364246,"redis_calls":13,"redis_duration_s":0.001753,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010243,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014955,"completed_at":"2026-10-04T19:09:03.803Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.932132] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2bb7a2839feabf599da0bd","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2bb7a2839feabf599da0bd: start","job_status":"start","queue_duration_s":1.380282,"scheduling_latency_s":1.3803} gitlab # [ 310.944934] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d2bb7a2839feabf599da0bd","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d2bb7a2839feabf599da0bd: done: 0.014373 sec","job_status":"done","queue_duration_s":1.380282,"scheduling_latency_s":1.3803,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014373,"completed_at":"2026-10-04T19:09:03.818Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.985128] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.819Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4ea2c08c0447aca3a1745eb","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4ea2c08c0447aca3a1745eb: start","job_status":"start","queue_duration_s":1.395757,"scheduling_latency_s":1.395775} gitlab # [ 310.997902] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.834Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4ea2c08c0447aca3a1745eb","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4ea2c08c0447aca3a1745eb: done: 0.014421 sec","job_status":"done","queue_duration_s":1.395757,"scheduling_latency_s":1.395775,"redis_calls":13,"redis_duration_s":0.00148,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014421,"completed_at":"2026-10-04T19:09:03.834Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 311.039428] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e66ebc119a3b054a264b684","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e66ebc119a3b054a264b684: start","job_status":"start","queue_duration_s":1.41136,"scheduling_latency_s":1.411378} gitlab # [ 311.052356] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6e66ebc119a3b054a264b684","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6e66ebc119a3b054a264b684: done: 0.014499 sec","job_status":"done","queue_duration_s":1.41136,"scheduling_latency_s":1.411378,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014499,"completed_at":"2026-10-04T19:09:03.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 311.092795] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9571ab39b53578c3bdc2477b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9571ab39b53578c3bdc2477b: start","job_status":"start","queue_duration_s":1.426954,"scheduling_latency_s":1.426972} gitlab # [ 311.105582] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9571ab39b53578c3bdc2477b","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9571ab39b53578c3bdc2477b: done: 0.014363 sec","job_status":"done","queue_duration_s":1.426954,"scheduling_latency_s":1.426972,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014363,"completed_at":"2026-10-04T19:09:03.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 311.145968] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.866Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5084707771150f4c436e1c3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5084707771150f4c436e1c3: start","job_status":"start","queue_duration_s":1.442444,"scheduling_latency_s":1.442462} gitlab # [ 311.158845] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a5084707771150f4c436e1c3","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a5084707771150f4c436e1c3: done: 0.014417 sec","job_status":"done","queue_duration_s":1.442444,"scheduling_latency_s":1.442462,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014417,"completed_at":"2026-10-04T19:09:03.880Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 311.199344] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5966d646fbc0741bdc817faf","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5966d646fbc0741bdc817faf: start","job_status":"start","queue_duration_s":1.457994,"scheduling_latency_s":1.458011} gitlab # [ 311.212159] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:03.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5966d646fbc0741bdc817faf","created_at":"2026-10-04T19:09:02.347Z","trace_propagation_headers":{"sentry-trace":"d79cdf3de51449a1852df624349d76d9-7296e507ff204fb4","baggage":"sentry-trace_id=d79cdf3de51449a1852df624349d76d9,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"2f4b76a5d1f71cffd06d31bb17625565","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:02.424Z","job_size_bytes":2,"pid":1904,"sidekiq_tid":"mal4","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5966d646fbc0741bdc817faf: done: 0.014513 sec","job_status":"done","queue_duration_s":1.457994,"scheduling_latency_s":1.458011,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014513,"completed_at":"2026-10-04T19:09:03.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 315.649186] postgres[3373]: [3373] ERROR: relation "table_without_models" does not exist gitlab # [ 315.649590] postgres[3373]: [3373] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 321.433884] runuser[3022]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 321.435109] setup-gitlab-start[2987]: + systemctl restart gitlab-sidekiq.service gitlab # [ 321.460594] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 321.471679] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:16.393Z","message":"Shutting down"} gitlab # [ 321.475641] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:16.397Z","message":"Scheduler exiting..."} gitlab # [ 321.478074] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:16.398Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 321.480832] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:16.398Z","message":"Scheduler exiting..."} gitlab # [ 321.977241] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:16.899Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 324.480377] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:19.401Z","message":"Bye!"} gitlab # [ 324.506398] sidekiq-cluster[1904]: {"severity":"INFO","time":"2026-10-04T19:09:19.428Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1904,"worker_id":"sidekiq_0","memwd_rss_bytes":933298176,"retry":0} gitlab # [ 325.466567] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 325.466829] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 325.466969] systemd[1]: gitlab-sidekiq.service: Consumed 2min 8.936s CPU time over 3min 41.290s wall clock time, 937M memory peak, 4.5M read from disk. gitlab # [ 325.474363] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 325.476930] setup-gitlab-start[2987]: + mkdir -p /var/lib/secrets gitlab # [ 325.479914] setup-gitlab-start[2987]: + echo nixbot-test-token-of-20-chars gitlab # [ 325.481527] setup-gitlab-start[2987]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 325.483054] setup-gitlab-start[2987]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 325.948942] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:20 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 441 gitlab # [ 326.672869] gitaly[923]: time="2026-10-04T19:09:21.594Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.593" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.673198] gitaly[923]: time="2026-10-04T19:09:21.594Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.593" grpc.time_ms=0.551 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.828558] gitaly[923]: time="2026-10-04T19:09:21.750Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.159" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.750" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.846224] gitaly[923]: time="2026-10-04T19:09:21.751Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.159" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.750" grpc.time_ms=0.5 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.879158] gitaly[923]: time="2026-10-04T19:09:21.801Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN diskcache=c0112a9c-b922-42eb-9467-9f94a920c314 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T19:09:31.174" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.764" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.896076] gitaly[923]: time="2026-10-04T19:09:21.801Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=20 command.spawn_token_fork_ms=4 command.system_time_ms=10 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T19:09:31.174" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.764" grpc.time_ms=36.593 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.915966] gitaly[923]: time="2026-10-04T19:09:21.812Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.812" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 326.930155] gitaly[923]: time="2026-10-04T19:09:21.812Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.221" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.812" grpc.time_ms=0.655 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.944213] gitaly[923]: time="2026-10-04T19:09:21.827Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:31.227" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.818" grpc.time_ms=9.582 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.992149] gitaly[923]: time="2026-10-04T19:09:21.914Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.913" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.008245] gitaly[923]: time="2026-10-04T19:09:21.914Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.003" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.913" grpc.time_ms=0.541 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.026529] gitaly[923]: time="2026-10-04T19:09:21.946Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN diskcache=b02dfeb9-d4e3-413c-8bdd-826f71ef6b6a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T19:09:31.006" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.916" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.041740] gitaly[923]: time="2026-10-04T19:09:21.946Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1439 command.oublock=32 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=9 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-04T19:09:31.006" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.916" grpc.time_ms=29.644 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.059025] gitaly[923]: time="2026-10-04T19:09:21.952Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.041" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.951" pid=923 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 327.071304] gitaly[923]: time="2026-10-04T19:09:21.952Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:31.041" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.951" grpc.time_ms=0.708 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.083608] gitaly[923]: time="2026-10-04T19:09:21.964Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:31.046" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:21.956" grpc.time_ms=7.385 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.098518] gitaly[923]: time="2026-10-04T19:09:22.001Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:09:32.001" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:22.001" grpc.time_ms=0.193 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.110066] gitaly[923]: time="2026-10-04T19:09:22.011Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:09:32.010" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:22.011" grpc.time_ms=0.416 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.754289] gitaly[923]: time="2026-10-04T19:09:22.675Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44546KP5PFXB65BA5YDNHPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:32.074" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:22.665" grpc.time_ms=9.818 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 328.405258] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:23 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 2440 gitlab # [ 328.407344] setup-gitlab-start[3444]: {"id":1,"message":null,"error":null} gitlab # [ 330.677155] sidekiq-cluster[3427]: {"severity":"INFO","time":"2026-10-04T19:09:25.569Z","message":"Starting cluster with 1 processes"} gitlab # [ 333.552083] gitaly[923]: time="2026-10-04T19:09:28.473Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M4454DX0GCE8GFK4749PNXCG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:38.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:28.463" grpc.time_ms=9.841 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 333.630162] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:28 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 199 gitlab # [ 333.637482] setup-gitlab-start[2987]: + mkdir -p /tmp/test-flake gitlab # [ 333.640186] setup-gitlab-start[2987]: + cd /tmp/test-flake gitlab # [ 333.641423] setup-gitlab-start[2987]: + git init -b master gitlab # [ 333.663627] setup-gitlab-start[3517]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 333.665732] setup-gitlab-start[2987]: + git config user.name test gitlab # [ 333.667776] setup-gitlab-start[2987]: + git config user.email test@example.com gitlab # [ 333.671790] setup-gitlab-start[2987]: + cat gitlab # [ 333.675009] setup-gitlab-start[2987]: + git add flake.nix gitlab # [ 333.679767] setup-gitlab-start[2987]: + git commit -m 'initial commit' gitlab # [ 333.703891] setup-gitlab-start[3522]: [master (root-commit) 760c67a] initial commit gitlab # [ 333.706298] setup-gitlab-start[3522]: 1 file changed, 10 insertions(+) gitlab # [ 333.707866] setup-gitlab-start[3522]: create mode 100644 flake.nix gitlab # [ 333.709397] setup-gitlab-start[2987]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 333.711954] setup-gitlab-start[2987]: + git push -u origin master gitlab # [ 334.371318] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:29 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 630 gitlab # [ 334.500031] gitaly[923]: time="2026-10-04T19:09:29.422Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4454ETE463HHWXDQVGBZB16 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:29.410" grpc.time_ms=11.321 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 334.516298] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:29 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 128 gitlab # [ 335.062451] gitaly[923]: time="2026-10-04T19:09:29.983Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:39.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:29.972" grpc.time_ms=11.369 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.111768] gitaly[923]: time="2026-10-04T19:09:30.033Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T19:10:00.022" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T19:09:30.022" grpc.time_ms=10.954 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.153907] gitaly[923]: time="2026-10-04T19:09:30.075Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 duration_ms=501ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 335.201421] gitaly[923]: time="2026-10-04T19:09:30.123Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 duration_ms=46ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 335.208799] gitaly[923]: time="2026-10-04T19:09:30.123Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:29.573" grpc.time_ms=550.427 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.257537] gitaly[923]: time="2026-10-04T19:09:30.179Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:30.178" grpc.time_ms=0.951 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.305530] gitaly[923]: time="2026-10-04T19:09:30.227Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:30.226" grpc.time_ms=0.664 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.347336] gitaly[923]: time="2026-10-04T19:09:30.269Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:30.268" grpc.time_ms=0.464 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.519980] gitaly[923]: time="2026-10-04T19:09:30.442Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:09:40.001" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:30.432" grpc.time_ms=9.691 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.557696] gitaly[923]: time="2026-10-04T19:09:30.479Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T19:09:40.024" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:30.454" grpc.time_ms=24.914 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.579989] gitaly[923]: time="2026-10-04T19:09:30.496Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 duration_ms=198ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 335.584859] gitaly[923]: time="2026-10-04T19:09:30.496Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:30.296" grpc.time_ms=199.908 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.602174] gitlab-workhorse[822]: time="2026-10-04T19:09:30Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 335.605184] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:30 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1070 gitlab # [ 335.608490] systemd[1]: Finished setup-gitlab.service. gitlab # [ 335.609705] setup-gitlab-start[3526]: To http://gitlab/root/test-flake.git gitlab # [ 335.611032] setup-gitlab-start[3526]: * [new branch] master -> master gitlab # [ 335.612357] setup-gitlab-start[3526]: branch 'master' set up to track 'origin/master'. gitlab # [ 335.613808] gitaly[923]: time="2026-10-04T19:09:30.504Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 diskcache=479ad373-3f01-4099-b802-99f4362f5cb6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:29.504" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.626882] systemd[1]: Starting nixbot CI service... gitlab # [ 335.627845] gitaly[923]: time="2026-10-04T19:09:30.505Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=205 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13615 command.oublock=32 command.real_time_ms=994 command.spawn_token_fork_ms=0 command.system_time_ms=111 command.user_time_ms=94 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:29.504" grpc.time_ms=1000.49 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab: (finished: waiting for unit setup-gitlab.service, in 336.70 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 336.845344] nixbot[3615]: {"ts": "2026-10-04T19:09:31.766Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 336.871996] nixbot[3615]: {"ts": "2026-10-04T19:09:31.793Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 336.895550] nixbot[3615]: {"ts": "2026-10-04T19:09:31.817Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 336.899980] nixbot[3615]: {"ts": "2026-10-04T19:09:31.822Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 336.904207] nixbot[3615]: {"ts": "2026-10-04T19:09:31.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 336.910412] nixbot[3615]: {"ts": "2026-10-04T19:09:31.832Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 336.914491] nixbot[3615]: {"ts": "2026-10-04T19:09:31.836Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 336.918029] nixbot[3615]: {"ts": "2026-10-04T19:09:31.838Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 336.922093] nixbot[3615]: {"ts": "2026-10-04T19:09:31.844Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 336.925580] nixbot[3615]: {"ts": "2026-10-04T19:09:31.845Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 336.929113] nixbot[3615]: {"ts": "2026-10-04T19:09:31.850Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 336.935025] nixbot[3615]: {"ts": "2026-10-04T19:09:31.856Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 336.938777] nixbot[3615]: {"ts": "2026-10-04T19:09:31.860Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 336.942425] nixbot[3615]: {"ts": "2026-10-04T19:09:31.862Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 336.946194] nixbot[3615]: {"ts": "2026-10-04T19:09:31.865Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 336.949830] nixbot[3615]: {"ts": "2026-10-04T19:09:31.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 336.953238] nixbot[3615]: {"ts": "2026-10-04T19:09:31.869Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 336.956701] nixbot[3615]: {"ts": "2026-10-04T19:09:31.870Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 336.959920] nixbot[3615]: {"ts": "2026-10-04T19:09:31.874Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 336.963556] nixbot[3615]: {"ts": "2026-10-04T19:09:31.875Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 336.967215] nixbot[3615]: {"ts": "2026-10-04T19:09:31.877Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 336.970760] nixbot[3615]: {"ts": "2026-10-04T19:09:31.882Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 336.974380] nixbot[3615]: {"ts": "2026-10-04T19:09:31.883Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 336.978628] nixbot[3615]: {"ts": "2026-10-04T19:09:31.885Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 336.981857] nixbot[3615]: {"ts": "2026-10-04T19:09:31.886Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 336.985083] nixbot[3615]: {"ts": "2026-10-04T19:09:31.888Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 336.988191] nixbot[3615]: {"ts": "2026-10-04T19:09:31.890Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 336.991387] nixbot[3615]: {"ts": "2026-10-04T19:09:31.891Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 336.994485] nixbot[3615]: {"ts": "2026-10-04T19:09:31.893Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 336.997460] nixbot[3615]: {"ts": "2026-10-04T19:09:31.906Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 337.000387] nixbot[3615]: {"ts": "2026-10-04T19:09:31.911Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 337.003141] nixbot[3615]: {"ts": "2026-10-04T19:09:31.912Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 337.006135] nixbot[3615]: {"ts": "2026-10-04T19:09:31.913Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 337.009476] nixbot[3615]: {"ts": "2026-10-04T19:09:31.916Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 337.012331] nixbot[3615]: {"ts": "2026-10-04T19:09:31.918Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 337.014969] nixbot[3615]: {"ts": "2026-10-04T19:09:31.923Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 337.017737] nixbot[3615]: {"ts": "2026-10-04T19:09:31.925Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"} gitlab # [ 337.503928] nixbot[3615]: INFO: Started server process [3615] gitlab # [ 337.505550] nixbot[3615]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 337.531900] nixbot[3615]: INFO: ::ffff:127.0.0.1:44068 - "GET /health HTTP/1.1" 200 OK gitlab # [ 337.536300] systemd[1]: Started nixbot CI service. gitlab # [ 337.537775] systemd[1]: Reached target Multi-User System. gitlab # [ 337.539139] systemd[1]: Startup finished in 670ms (kernel) + 3.633s (initrd) + 5min 33.234s (userspace) = 5min 37.539s. gitlab: (finished: waiting for unit nixbot.service, in 2.08 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 338.389272] nixbot[3615]: INFO: ::ffff:127.0.0.1:44072 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 338.454603] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 926 gitlab # [ 338.458711] nixbot[3615]: {"ts": "2026-10-04T19:09:33.377Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 338.484317] nixbot[3615]: {"ts": "2026-10-04T19:09:33.406Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.12 seconds) gitlab # [ 338.511274] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 85 gitlab # [ 338.576589] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 81 gitlab # [ 338.582998] nixbot[3615]: {"ts": "2026-10-04T19:09:33.501Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 338.586573] nixbot[3615]: {"ts": "2026-10-04T19:09:33.503Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 338.712529] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 917 "" "python-httpx/0.28.1" 128 gitlab # [ 338.714780] nixbot[3615]: {"ts": "2026-10-04T19:09:33.635Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 338.868752] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 145 gitlab # [ 338.871308] nixbot[3615]: {"ts": "2026-10-04T19:09:33.792Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 339.005581] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "python-httpx/0.28.1" 125 gitlab # [ 339.009188] nixbot[3615]: {"ts": "2026-10-04T19:09:33.928Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 339.078957] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 917 "" "python-httpx/0.28.1" 68 gitlab # [ 339.082636] nixbot[3615]: {"ts": "2026-10-04T19:09:34.002Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 339.176463] gitaly[923]: time="2026-10-04T19:09:34.095Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KDRK8XBR2HFM2X16AB3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-04T19:10:04.079" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:34.079" grpc.time_ms=15.461 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.200178] gitaly[923]: time="2026-10-04T19:09:34.119Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KDRK8XBR2HFM2X16AB3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:04.117" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:34.118" grpc.time_ms=1.088 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.216223] gitaly[923]: time="2026-10-04T19:09:34.134Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KDRK8XBR2HFM2X16AB3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-04T19:09:44.003" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:34.123" grpc.time_ms=11.028 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.246402] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 160 gitlab # [ 339.249577] nixbot[3615]: {"ts": "2026-10-04T19:09:34.169Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 339.346456] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 94 gitlab # [ 339.350587] nixbot[3615]: {"ts": "2026-10-04T19:09:34.269Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 339.355098] nixbot[3615]: {"ts": "2026-10-04T19:09:34.272Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 339.501417] gitaly[923]: time="2026-10-04T19:09:34.423Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454KRA0WY471XG4TF9PK49 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.413" grpc.time_ms=10.294 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.527332] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 76 gitlab # [ 339.605258] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 880 "" "curl/8.21.0" 62 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.10 seconds) (finished: subtest: project discovered and webhook registered, in 1.22 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 339.625101] gitaly[923]: time="2026-10-04T19:09:34.546Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4454KTSXP61WWX155R21426 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.531" pid=923 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 339.642075] gitaly[923]: time="2026-10-04T19:09:34.547Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KTSXP61WWX155R21426 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.531" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.663904] gitlab-workhorse[822]: time="2026-10-04T19:09:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 339.667298] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 124 gitlab # [ 339.670571] gitaly[923]: time="2026-10-04T19:09:34.547Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KTSXP61WWX155R21426 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.531" grpc.time_ms=15.337 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.728474] gitaly[923]: time="2026-10-04T19:09:34.650Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4454KYSJ2PGGK4BX30DT3SS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.631" pid=923 remote_ip="2001:db8:1::1" request_sha=7e0ce7f2005165a69796d2013ca14579702b16eb response_bytes=223 span.kind=server system=grpc gitlab # [ 339.745671] gitaly[923]: time="2026-10-04T19:09:34.650Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KYSJ2PGGK4BX30DT3SS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.631" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.767402] gitlab-workhorse[822]: time="2026-10-04T19:09:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=223 gitlab # [ 339.770212] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 98 gitlab # [ 339.772843] gitaly[923]: time="2026-10-04T19:09:34.650Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454KYSJ2PGGK4BX30DT3SS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.631" grpc.time_ms=19.662 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.840356] gitaly[923]: time="2026-10-04T19:09:34.762Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=5 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454M34EC053KYRH726RYT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.748" grpc.time_ms=13.595 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.861561] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 70 gitlab # [ 339.919752] gitaly[923]: time="2026-10-04T19:09:34.841Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4454M5CSRNZ8ERWT7SSGV45 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.831" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 339.936531] gitaly[923]: time="2026-10-04T19:09:34.841Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454M5CSRNZ8ERWT7SSGV45 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.831" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 339.957559] gitlab-workhorse[822]: time="2026-10-04T19:09:34Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 339.960257] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 78 gitlab # [ 339.963569] gitaly[923]: time="2026-10-04T19:09:34.842Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454M5CSRNZ8ERWT7SSGV45 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.831" grpc.time_ms=10.694 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.033728] gitaly[923]: time="2026-10-04T19:09:34.955Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454M967DV8MRDKXQEPCYEM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:34.945" grpc.time_ms=10.265 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.056340] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:34 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 70 gitlab # [ 340.135272] gitaly[923]: time="2026-10-04T19:09:35.056Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4454MBEKBAD6ANCA7WZ4E93 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:35.038" pid=923 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 340.152361] gitaly[923]: time="2026-10-04T19:09:35.057Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MBEKBAD6ANCA7WZ4E93 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:35.038" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.175452] gitaly[923]: time="2026-10-04T19:09:35.074Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MBEKBAD6ANCA7WZ4E93 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:35.038" grpc.time_ms=18.789 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.197893] gitlab-workhorse[822]: time="2026-10-04T19:09:35Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 340.200603] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:35 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 115 gitlab # [ 340.459511] gitaly[923]: time="2026-10-04T19:09:35.381Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:05.380" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:35.380" grpc.time_ms=0.703 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.525023] gitaly[923]: time="2026-10-04T19:09:35.447Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T19:10:05.435" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:35.436" grpc.time_ms=10.833 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.575851] gitaly[923]: time="2026-10-04T19:09:35.497Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:09:45.057" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:35.488" grpc.time_ms=9.585 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.597922] gitaly[923]: time="2026-10-04T19:09:35.519Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:09:45.078" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:35.509" grpc.time_ms=10.041 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.725188] gitaly[923]: time="2026-10-04T19:09:35.647Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:09:45.216" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:35.646" grpc.time_ms=0.387 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.759075] gitaly[923]: time="2026-10-04T19:09:35.681Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:09:45.240" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:35.671" grpc.time_ms=9.521 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.812181] gitaly[923]: time="2026-10-04T19:09:35.734Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:35.733" grpc.time_ms=0.535 pid=923 span.kind=server system=grpc gitlab # [ 340.841662] gitaly[923]: time="2026-10-04T19:09:35.763Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:09:35.763" grpc.time_ms=0.633 pid=923 span.kind=server system=grpc gitlab # [ 340.857769] gitaly[923]: time="2026-10-04T19:09:35.771Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q diskcache=583d3ace-7ad1-4291-aca5-1753ab2ee803 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:09:45.253" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:35.684" pid=923 ref=refs/keep-around/760c67a24da4b813ae4e4eb76ca01815d143bd41 remote_ip="2001:db8:1::1" revision=760c67a24da4b813ae4e4eb76ca01815d143bd41 span.kind=server system=grpc username=root gitlab # [ 340.876763] gitaly[923]: time="2026-10-04T19:09:35.771Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=76 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5292 command.oublock=8 command.real_time_ms=82 command.spawn_token_fork_ms=0 command.system_time_ms=43 command.user_time_ms=33 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:09:45.253" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:09:35.684" grpc.time_ms=87.146 pid=923 ref=refs/keep-around/760c67a24da4b813ae4e4eb76ca01815d143bd41 remote_ip="2001:db8:1::1" revision=760c67a24da4b813ae4e4eb76ca01815d143bd41 span.kind=server system=grpc username=root gitlab # [ 340.896386] gitaly[923]: time="2026-10-04T19:09:35.777Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:05.776" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:35.777" grpc.time_ms=0.45 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.053579] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:35 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 859 gitlab # [ 341.058587] nixbot[3615]: {"ts": "2026-10-04T19:09:35.976Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 341.064636] nixbot[3615]: {"ts": "2026-10-04T19:09:35.986Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-760c67a24da4-0"} gitlab # [ 341.282156] nixbot[3615]: {"ts": "2026-10-04T19:09:36.204Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-760c67a24da4-0", "argv0": "bwrap"} gitlab # [ 341.433684] systemd[1]: Started Nix Daemon. gitlab # [ 341.513192] nix-daemon[3777]: accepted connection from pid 3772, user nixbot gitlab # [ 345.956862] nix-daemon[3777]: accepted connection from pid 3823, user nixbot gitlab # [ 345.972242] nix-daemon[3777]: accepted connection from pid 3772, user nixbot gitlab # [ 350.952381] nix-daemon[3777]: accepted connection from pid 3772, user nixbot gitlab # [ 351.092373] gitaly[923]: time="2026-10-04T19:09:46.013Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454Z29ZFFTY1K5QAQET6J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:16.012" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:46.012" grpc.time_ms=0.867 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.140802] gitaly[923]: time="2026-10-04T19:09:46.062Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4454Z29ZFFTY1K5QAQET6J9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:09:56.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:46.052" grpc.time_ms=10.511 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.208577] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 199 gitlab # [ 351.214450] nixbot[3615]: {"ts": "2026-10-04T19:09:46.131Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 351.266346] gitaly[923]: time="2026-10-04T19:09:46.188Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454Z8QF6MDKJGTQBXMRC3N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:16.186" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:46.187" grpc.time_ms=0.726 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.309167] gitaly[923]: time="2026-10-04T19:09:46.231Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4454Z8QF6MDKJGTQBXMRC3N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:09:56.102" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:46.222" grpc.time_ms=8.318 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.373633] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 159 gitlab # [ 351.377878] nixbot[3615]: {"ts": "2026-10-04T19:09:46.296Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 351.467240] gitaly[923]: time="2026-10-04T19:09:46.389Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454ZE42K250QS9D5PD0NSV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:10:16.087" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:09:46.388" grpc.time_ms=0.782 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.671035] gitaly[923]: time="2026-10-04T19:09:46.593Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4454ZE42K250QS9D5PD0NSV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:09:56.122" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:09:46.582" grpc.time_ms=10.461 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 351.820074] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 433 gitlab # [ 351.823755] nixbot[3615]: {"ts": "2026-10-04T19:09:46.743Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 352.203894] gitaly[923]: time="2026-10-04T19:09:47.125Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M445505JMS9ABDF2AH7AVNSP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.115" grpc.time_ms=9.463 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 352.218535] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 68 gitlab # [ 352.293288] gitaly[923]: time="2026-10-04T19:09:47.215Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M445507RJRSZRFN7Z5ERNQDK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.204" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 352.310053] gitaly[923]: time="2026-10-04T19:09:47.215Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445507RJRSZRFN7Z5ERNQDK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.204" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 352.331314] gitlab-workhorse[822]: time="2026-10-04T19:09:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 352.335033] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 88 gitlab # [ 352.337784] gitaly[923]: time="2026-10-04T19:09:47.215Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445507RJRSZRFN7Z5ERNQDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.204" grpc.time_ms=11.629 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 352.403263] gitaly[923]: time="2026-10-04T19:09:47.325Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M44550BQHWG6KKS3BRJS54HC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.315" grpc.time_ms=9.102 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 352.424633] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 352.481069] gitaly[923]: time="2026-10-04T19:09:47.403Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M44550DZ9NKP8V7YFA58EZZ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.392" pid=923 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 352.494254] gitaly[923]: time="2026-10-04T19:09:47.403Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M44550DZ9NKP8V7YFA58EZZ4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.392" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 352.509310] gitlab-workhorse[822]: time="2026-10-04T19:09:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 352.511253] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:09:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 76 gitlab # [ 352.513344] gitaly[923]: time="2026-10-04T19:09:47.403Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M44550DZ9NKP8V7YFA58EZZ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:09:47.392" grpc.time_ms=10.43 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 373.058538] sidekiq-cluster[3494]: 2026-10-04T19:10:07.979Z pid=3494 tid=yku INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3494, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>431583232} gitlab # [ 373.359320] sidekiq-cluster[3494]: W, [2026-10-04T19:10:08.281314 #3494] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 373.425751] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:08.347Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 376.966861] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:11.888Z","message":"GitLab reliable fetch activated!"} gitlab # [ 376.967141] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:11.888Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 413.262944] postgres[4383]: [4383] ERROR: relation "table_without_models" does not exist gitlab # [ 413.263344] postgres[4383]: [4383] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 418.041029] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:52.962Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 418.044726] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:52.966Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 418.046900] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:52.966Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 418.048803] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:52.966Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 418.050840] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:52.968Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 78.74 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 418.475015] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:10:53 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 76 gitlab # [ 418.649572] gitaly[923]: time="2026-10-04T19:10:53.571Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44570YN9H65Q0QHKPF0MP9D grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:10:53.560" grpc.time_ms=10.122 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 418.671692] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:10:53 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 201 "" "git/2.55.0" 173 gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 418.965308] gitaly[923]: time="2026-10-04T19:10:53.887Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-04T19:11:23.575" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-04T19:10:53.875" grpc.time_ms=11.05 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.995765] gitaly[923]: time="2026-10-04T19:10:53.917Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M445715A32X6Y3Q1M1V3H3VX duration_ms=148ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 419.032030] gitaly[923]: time="2026-10-04T19:10:53.954Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M445715A32X6Y3Q1M1V3H3VX duration_ms=35ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 419.039195] gitaly[923]: time="2026-10-04T19:10:53.955Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:53.768" grpc.time_ms=186.417 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 419.091796] gitaly[923]: time="2026-10-04T19:10:54.013Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:54.009" grpc.time_ms=0.819 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 419.137443] gitaly[923]: time="2026-10-04T19:10:54.059Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:54.058" grpc.time_ms=0.638 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 419.185992] gitaly[923]: time="2026-10-04T19:10:54.108Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:54.107" grpc.time_ms=0.622 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 419.541845] gitaly[923]: time="2026-10-04T19:10:54.463Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M445715A32X6Y3Q1M1V3H3VX duration_ms=322ns method=POST pid=923 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 419.549086] gitaly[923]: time="2026-10-04T19:10:54.464Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:54.139" grpc.time_ms=324.503 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 419.567506] gitaly[923]: time="2026-10-04T19:10:54.489Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX diskcache=085715f6-59af-4c7f-b7a1-188e33bb992c grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:10:53.704" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 419.584419] gitaly[923]: time="2026-10-04T19:10:54.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=213 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13377 command.oublock=32 command.real_time_ms=763 command.spawn_token_fork_ms=0 command.system_time_ms=103 command.user_time_ms=110 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:10:53.704" grpc.time_ms=785.527 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # 760c67a..459fc93 master -> master gitlab # [ 419.608183] gitlab-workhorse[822]: time="2026-10-04T19:10:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 419.611224] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:10:54 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 880 gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.27 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.10 seconds) gitlab # [ 419.729530] nixbot[3615]: INFO: ::ffff:127.0.0.1:39528 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.04 seconds) gitlab # [ 420.761824] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:55.682Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d203d38b0598a94b1e04f556","created_at":"2026-10-04T19:09:21.840Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:21.884Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d203d38b0598a94b1e04f556: start","job_status":"start","queue_duration_s":93.797911,"scheduling_latency_s":93.797963} gitlab # [ 420.772657] nixbot[3615]: INFO: ::ffff:127.0.0.1:39544 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab # [ 420.942402] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:55.863Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"d203d38b0598a94b1e04f556","created_at":"2026-10-04T19:09:21.840Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:21.884Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-d203d38b0598a94b1e04f556: done: 0.181167 sec","job_status":"done","queue_duration_s":93.797911,"scheduling_latency_s":93.797963,"redis_calls":23,"redis_duration_s":0.004471,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002348,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.086645,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09743966,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.181167,"completed_at":"2026-10-04T19:10:55.863Z","load_balancing_strategy":"primary","db_duration_s":0.057534,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 421.014704] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:55.865Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"91117a04959466897c3c71bd","created_at":"2026-10-04T19:09:21.901Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:21.908Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-91117a04959466897c3c71bd: start","job_status":"start","queue_duration_s":93.957203,"scheduling_latency_s":93.957222} gitlab # [ 421.044689] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:55.966Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"91117a04959466897c3c71bd","created_at":"2026-10-04T19:09:21.901Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:21.908Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-91117a04959466897c3c71bd: done: 0.100838 sec","job_status":"done","queue_duration_s":93.957203,"scheduling_latency_s":93.957222,"redis_calls":14,"redis_duration_s":0.002584,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000533,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07767774,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.100838,"completed_at":"2026-10-04T19:10:55.966Z","load_balancing_strategy":"primary","db_duration_s":0.020513,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 421.110702] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:55.967Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"914fe64e5aef509f07eeee6d","created_at":"2026-10-04T19:09:22.520Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.528Z","job_size_bytes":5,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-914fe64e5aef509f07eeee6d: start","job_status":"start","queue_duration_s":93.439127,"scheduling_latency_s":93.439144} gitlab # [ 421.207643] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.129Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"914fe64e5aef509f07eeee6d","created_at":"2026-10-04T19:09:22.520Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.528Z","job_size_bytes":5,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-914fe64e5aef509f07eeee6d: done: 0.161708 sec","job_status":"done","queue_duration_s":93.439127,"scheduling_latency_s":93.439144,"redis_calls":12,"redis_duration_s":0.002198,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001272,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.131523,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025628,"duration_s":0.161708,"completed_at":"2026-10-04T19:10:56.129Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025475,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 421.277914] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.130Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8460d4d97f71e97aa35dabaa","created_at":"2026-10-04T19:09:22.560Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.565Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8460d4d97f71e97aa35dabaa: start","job_status":"start","queue_duration_s":93.564854,"scheduling_latency_s":93.564872} gitlab # [ 421.414341] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.336Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"8460d4d97f71e97aa35dabaa","created_at":"2026-10-04T19:09:22.560Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.565Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-8460d4d97f71e97aa35dabaa: done: 0.205328 sec","job_status":"done","queue_duration_s":93.564854,"scheduling_latency_s":93.564872,"redis_calls":11,"redis_duration_s":0.002126,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000602,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001166,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000358,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.17,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.17,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.141818,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024506,"duration_s":0.205328,"completed_at":"2026-10-04T19:10:56.335Z","load_balancing_strategy":"primary","db_duration_s":0.066691,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 421.482668] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.337Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"eca645ac4aa871221e2bc8cf","created_at":"2026-10-04T19:09:22.569Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.576Z","job_size_bytes":89,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-eca645ac4aa871221e2bc8cf: start","job_status":"start","queue_duration_s":93.760884,"scheduling_latency_s":93.760901} gitlab # [ 421.505797] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.373Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"eca645ac4aa871221e2bc8cf","created_at":"2026-10-04T19:09:22.569Z","trace_propagation_headers":{"sentry-trace":"ff62fbac685a45119db1713962b1d174-6fd261152219446b","baggage":"sentry-trace_id=ff62fbac685a45119db1713962b1d174,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44546KP5PFXB65BA5YDNHPN","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:22.576Z","job_size_bytes":89,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-eca645ac4aa871221e2bc8cf: done: 0.036371 sec","job_status":"done","queue_duration_s":93.760884,"scheduling_latency_s":93.760901,"redis_calls":11,"redis_duration_s":0.001931,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000503,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001057,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000371,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025011,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027394,"duration_s":0.036371,"completed_at":"2026-10-04T19:10:56.373Z","load_balancing_strategy":"primary","db_duration_s":0.005021,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 421.556813] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.375Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 760c67a24da4b813ae4e4eb76ca01815d143bd41 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"88efaadebcd65963cab17da1","created_at":"2026-10-04T19:09:30.351Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-d77565acb716469e","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:30.388Z","job_size_bytes":185,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-88efaadebcd65963cab17da1: start","job_status":"start","queue_duration_s":85.986672,"scheduling_latency_s":85.986689} gitlab # [ 421.574172] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:56.390Z","message":"changes: \"0000000000000000000000000000000000000000 760c67a24da4b813ae4e4eb76ca01815d143bd41 refs/heads/master\\n\""} gitlab # [ 421.577085] gitaly[923]: time="2026-10-04T19:10:56.427Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:06.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:56.417" grpc.time_ms=10.035 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.590794] gitaly[923]: time="2026-10-04T19:10:56.441Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:11:06.021" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:56.432" grpc.time_ms=8.863 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.604733] gitaly[923]: time="2026-10-04T19:10:56.495Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T19:11:06.074" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:56.485" grpc.time_ms=10.405 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.618392] gitaly[923]: time="2026-10-04T19:10:56.538Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:56.538" grpc.time_ms=0.553 pid=923 span.kind=server system=grpc gitlab # [ 421.643015] gitaly[923]: time="2026-10-04T19:10:56.565Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:56.564" grpc.time_ms=0.496 pid=923 span.kind=server system=grpc gitlab # [ 421.654742] gitaly[923]: time="2026-10-04T19:10:56.571Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 diskcache=ba2922f6-9048-4949-8bb7-3bd61ab4565f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:11:06.097" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:56.507" pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 421.665988] gitaly[923]: time="2026-10-04T19:10:56.571Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=67 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=4949 command.oublock=8 command.real_time_ms=60 command.spawn_token_fork_ms=0 command.system_time_ms=59 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:11:06.097" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:56.507" grpc.time_ms=64.419 pid=923 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 421.681596] gitaly[923]: time="2026-10-04T19:10:56.587Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-04T19:11:06.176" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:56.586" grpc.time_ms=0.193 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.692724] gitaly[923]: time="2026-10-04T19:10:56.603Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-04T19:11:06.180" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:56.591" grpc.time_ms=12.493 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 421.778837] gitaly[923]: time="2026-10-04T19:10:56.700Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-04T19:11:26.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:56.694" grpc.time_ms=6.544 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.805837] nixbot[3615]: INFO: ::ffff:127.0.0.1:43196 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.04 seconds) gitlab # [ 421.816172] gitaly[923]: time="2026-10-04T19:10:56.738Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:11:06.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:56.737" grpc.time_ms=0.319 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.833252] gitaly[923]: time="2026-10-04T19:10:56.746Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T19:11:06.010" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T19:10:56.740" grpc.time_ms=6.29 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.852315] gitaly[923]: time="2026-10-04T19:10:56.772Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:06.032" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:56.763" grpc.time_ms=9.747 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.868075] gitaly[923]: time="2026-10-04T19:10:56.788Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:06.049" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:56.779" grpc.time_ms=9.52 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.896683] gitaly[923]: time="2026-10-04T19:10:56.818Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:11:26.202" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:56.802" grpc.time_ms=16.317 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.433723] gitaly[923]: time="2026-10-04T19:10:57.355Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T19:11:07.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:57.354" grpc.time_ms=1.578 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.467671] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.388Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 760c67a24da4b813ae4e4eb76ca01815d143bd41 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"88efaadebcd65963cab17da1","created_at":"2026-10-04T19:09:30.351Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-d77565acb716469e","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:09:30.388Z","job_size_bytes":185,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-88efaadebcd65963cab17da1: done: 1.013256 sec","job_status":"done","queue_duration_s":85.986672,"scheduling_latency_s":85.986689,"gitaly_calls":13,"gitaly_duration_s":0.193037,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.038075,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.003581,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005623,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001326,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003739,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.013541,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.010265,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.051,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.278,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.306,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.051,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.738015,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005695,"exclusive_lock_hold_duration_s":0.028480039999999998,"duration_s":1.013256,"completed_at":"2026-10-04T19:10:57.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.09206,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 422.548250] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.390Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cfd4d16dc6426084a63d461a","created_at":"2026-10-04T19:09:35.787Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.794Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cfd4d16dc6426084a63d461a: start","job_status":"start","queue_duration_s":81.596247,"scheduling_latency_s":81.596274} gitlab # [ 422.568338] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.477Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"cfd4d16dc6426084a63d461a","created_at":"2026-10-04T19:09:35.787Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.794Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-cfd4d16dc6426084a63d461a: done: 0.087208 sec","job_status":"done","queue_duration_s":81.596247,"scheduling_latency_s":81.596274,"redis_calls":16,"redis_duration_s":0.003469,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001852,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001227,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00039,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.059275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025898,"duration_s":0.087208,"completed_at":"2026-10-04T19:10:57.477Z","load_balancing_strategy":"primary","db_duration_s":0.020934,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 422.615155] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.479Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"fea489d52e02232b881ae3ee","created_at":"2026-10-04T19:09:35.795Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.802Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-fea489d52e02232b881ae3ee: start","job_status":"start","queue_duration_s":81.676512,"scheduling_latency_s":81.676532} gitlab # [ 422.681721] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.603Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"fea489d52e02232b881ae3ee","created_at":"2026-10-04T19:09:35.795Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.802Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-fea489d52e02232b881ae3ee: done: 0.124119 sec","job_status":"done","queue_duration_s":81.676512,"scheduling_latency_s":81.676532,"redis_calls":21,"redis_duration_s":0.0051990000000000005,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002396,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00146,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001343,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00049734,"duration_s":0.124119,"completed_at":"2026-10-04T19:10:57.603Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.031246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.751944] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:57.605Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"03b5ba402ddd3e407953445e","created_at":"2026-10-04T19:09:35.947Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.953Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-03b5ba402ddd3e407953445e: start","job_status":"start","queue_duration_s":81.651235,"scheduling_latency_s":81.651258} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 422.844540] nixbot[3615]: INFO: ::ffff:127.0.0.1:43210 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.04 seconds) gitlab # [ 422.997119] gitaly[923]: time="2026-10-04T19:10:57.918Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:57.917" grpc.time_ms=0.676 pid=923 span.kind=server system=grpc gitlab # [ 423.029786] gitaly[923]: time="2026-10-04T19:10:57.951Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:57.951" grpc.time_ms=0.522 pid=923 span.kind=server system=grpc gitlab # [ 423.059926] gitaly[923]: time="2026-10-04T19:10:57.982Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:10:57.981" grpc.time_ms=0.541 pid=923 span.kind=server system=grpc gitlab # [ 423.076292] gitaly[923]: time="2026-10-04T19:10:57.989Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q diskcache=3e8fcedc-281f-4ee2-a85a-9ceff5ae8600 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:11:27.285" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:57.885" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.090844] gitaly[923]: time="2026-10-04T19:10:57.990Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=111 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7597 command.oublock=0 command.real_time_ms=99 command.spawn_token_fork_ms=0 command.system_time_ms=57 command.user_time_ms=54 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:11:27.285" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:57.885" grpc.time_ms=103.956 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.193152] gitaly[923]: time="2026-10-04T19:10:58.115Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:11:28.013" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.113" grpc.time_ms=1.185 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.243526] gitaly[923]: time="2026-10-04T19:10:58.165Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454MGB7MZ3KG17SRDMED0Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:28.064" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.164" grpc.time_ms=0.916 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.266277] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.187Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"03b5ba402ddd3e407953445e","created_at":"2026-10-04T19:09:35.947Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-082483d195ee4b4a","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:09:35.953Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-03b5ba402ddd3e407953445e: done: 0.582492 sec","job_status":"done","queue_duration_s":81.651235,"scheduling_latency_s":81.651258,"gitaly_calls":3,"gitaly_duration_s":0.113968,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.026025,"redis_read_bytes":455,"redis_write_bytes":27995,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001213,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002758,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008978,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.003369,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14490,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.004574,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000571,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":165,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000501,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002848,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.187,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.024,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.027,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.156,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.390811,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.38624087999999995,"duration_s":0.582492,"completed_at":"2026-10-04T19:10:58.187Z","load_balancing_strategy":"primary","db_duration_s":0.198145,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.342236] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.189Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","760c67a24da4b813ae4e4eb76ca01815d143bd41 459fc935a5411646cc3e2e58cd0153ee2c26ddcf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"87d12a57ee1a947452ca4b6d","created_at":"2026-10-04T19:10:54.413Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-7b7ccd5ebddd4720","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:10:54.420Z","job_size_bytes":185,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-87d12a57ee1a947452ca4b6d: start","job_status":"start","queue_duration_s":3.769156,"scheduling_latency_s":3.769182} gitlab # [ 423.361731] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.204Z","message":"changes: \"760c67a24da4b813ae4e4eb76ca01815d143bd41 459fc935a5411646cc3e2e58cd0153ee2c26ddcf refs/heads/master\\n\""} gitlab # [ 423.364466] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.253Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"036e20f2b56a4d29f6ccb3ab","created_at":"2026-10-04T19:10:58.251Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"4a7fc08125cd430b97a49ca7","job_size_bytes":46,"pid":3494,"job_status":"deduplicated","message":"ProjectCacheWorker JID-036e20f2b56a4d29f6ccb3ab: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 423.380977] gitaly[923]: time="2026-10-04T19:10:58.234Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:08.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:58.224" grpc.time_ms=10.159 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.394876] gitaly[923]: time="2026-10-04T19:10:58.249Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-04T19:11:08.019" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:58.240" grpc.time_ms=8.879 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.409164] gitaly[923]: time="2026-10-04T19:10:58.276Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-04T19:11:08.046" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:58.266" grpc.time_ms=9.53 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.422977] gitaly[923]: time="2026-10-04T19:10:58.290Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-04T19:11:28.180" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.280" grpc.time_ms=9.69 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.439104] gitaly[923]: time="2026-10-04T19:10:58.307Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=319 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=4 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-04T19:11:08.080" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-04T19:10:58.301" grpc.time_ms=5.744 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.453654] gitaly[923]: time="2026-10-04T19:10:58.327Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:08.097" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:58.317" grpc.time_ms=9.156 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.497495] gitaly[923]: time="2026-10-04T19:10:58.419Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-04T19:11:08.198" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.418" grpc.time_ms=1.126 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.524429] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.445Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","760c67a24da4b813ae4e4eb76ca01815d143bd41 459fc935a5411646cc3e2e58cd0153ee2c26ddcf refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"87d12a57ee1a947452ca4b6d","created_at":"2026-10-04T19:10:54.413Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-7b7ccd5ebddd4720","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:10:54.420Z","job_size_bytes":185,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-87d12a57ee1a947452ca4b6d: done: 0.255861 sec","job_status":"done","queue_duration_s":3.769156,"scheduling_latency_s":3.769182,"gitaly_calls":7,"gitaly_duration_s":0.070316,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.01956,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.001955,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000846,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003195,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007302,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.006262,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.024,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.024,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.16091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070606,"exclusive_lock_hold_duration_s":0.02096488,"duration_s":0.255861,"completed_at":"2026-10-04T19:10:58.445Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024002,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.579852] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.447Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"4a7fc08125cd430b97a49ca7","created_at":"2026-10-04T19:10:56.443Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.461Z","job_size_bytes":46,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4a7fc08125cd430b97a49ca7: start","job_status":"start","queue_duration_s":1.985983,"scheduling_latency_s":1.986007} gitlab # [ 423.596659] gitaly[923]: time="2026-10-04T19:10:58.505Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-05T01:10:58.499" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:10:58.500" grpc.time_ms=5.19 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.607705] gitaly[923]: time="2026-10-04T19:10:58.525Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T19:11:28.415" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.516" grpc.time_ms=8.597 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.635476] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.556Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"4a7fc08125cd430b97a49ca7","created_at":"2026-10-04T19:10:56.443Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.461Z","job_size_bytes":46,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-4a7fc08125cd430b97a49ca7: done: 0.109705 sec","job_status":"done","queue_duration_s":1.985983,"scheduling_latency_s":1.986007,"gitaly_calls":2,"gitaly_duration_s":0.018111,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.006279,"redis_read_bytes":69,"redis_write_bytes":4888,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001228,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.00055,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2116,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001411,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.00241,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.074751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032344,"duration_s":0.109705,"completed_at":"2026-10-04T19:10:58.556Z","load_balancing_strategy":"primary","db_duration_s":0.010928,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.686056] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.559Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c8a35ab3c63ff819b37faa98","created_at":"2026-10-04T19:10:56.477Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.481Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c8a35ab3c63ff819b37faa98: start","job_status":"start","queue_duration_s":2.077363,"scheduling_latency_s":2.077389} gitlab # [ 423.703444] gitaly[923]: time="2026-10-04T19:10:58.611Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 diskcache=b9993899-93e5-477c-9268-4427ce056669 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:10:58.587" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.588" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.714073] gitaly[923]: time="2026-10-04T19:10:58.612Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:10:58.587" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.588" grpc.time_ms=23.798 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.736277] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.657Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"c8a35ab3c63ff819b37faa98","created_at":"2026-10-04T19:10:56.477Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.481Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-c8a35ab3c63ff819b37faa98: done: 0.098853 sec","job_status":"done","queue_duration_s":2.077363,"scheduling_latency_s":2.077389,"gitaly_calls":1,"gitaly_duration_s":0.026326,"redis_calls":15,"redis_duration_s":0.003015,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000402,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001099,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000529,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.023,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.068679,"duration_s":0.098853,"completed_at":"2026-10-04T19:10:58.657Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015025,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.785465] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.659Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"7370c38f2a69101f40f759ab","created_at":"2026-10-04T19:10:56.581Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.584Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7370c38f2a69101f40f759ab: start","job_status":"start","queue_duration_s":2.07533,"scheduling_latency_s":2.075353} gitlab # [ 423.801647] gitaly[923]: time="2026-10-04T19:10:58.701Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-04T19:11:28.591" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.691" grpc.time_ms=10.006 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 423.815234] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.714Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"7370c38f2a69101f40f759ab","created_at":"2026-10-04T19:10:56.581Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:56.584Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-7370c38f2a69101f40f759ab: done: 0.054792 sec","job_status":"done","queue_duration_s":2.07533,"scheduling_latency_s":2.075353,"gitaly_calls":1,"gitaly_duration_s":0.012265,"redis_calls":16,"redis_duration_s":0.003517,"redis_read_bytes":52,"redis_write_bytes":2048,"redis_queues_calls":1,"redis_queues_duration_s":0.000336,"redis_queues_read_bytes":1,"redis_queues_write_bytes":979,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001582,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054792,"completed_at":"2026-10-04T19:10:58.714Z","load_balancing_strategy":"primary","db_duration_s":0.004127,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.861490] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:58.716Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2c41a85cf5769d4832007edd","created_at":"2026-10-04T19:10:57.167Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e0beb0d44f5447fcbab8839b6a369c169a22acadc55291301fde3ecfae4c5ea6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.174Z","job_size_bytes":467,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2c41a85cf5769d4832007edd: start","job_status":"start","queue_duration_s":1.542081,"scheduling_latency_s":1.542105} gitlab # [ 423.879377] gitaly[923]: time="2026-10-04T19:10:58.747Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:28.646" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.746" grpc.time_ms=0.663 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.05 seconds) gitlab # [ 423.892344] gitaly[923]: time="2026-10-04T19:10:58.759Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:11:08.019" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:58.750" grpc.time_ms=9.067 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.906116] nixbot[3615]: INFO: ::ffff:127.0.0.1:43212 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab # [ 423.990705] gitaly[923]: time="2026-10-04T19:10:58.912Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:11:28.811" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.911" grpc.time_ms=0.89 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.011882] gitaly[923]: time="2026-10-04T19:10:58.919Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=50 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T19:11:28.817" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:58.917" grpc.time_ms=2.437 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.113313] gitaly[923]: time="2026-10-04T19:10:59.035Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:09.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:10:59.026" grpc.time_ms=9.261 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.673823] gitaly[923]: time="2026-10-04T19:10:59.595Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:11:29.480" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:59.580" grpc.time_ms=15.155 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.691055] gitaly[923]: time="2026-10-04T19:10:59.613Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M4454EYVSDPR9PFBYBH0VGP9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T19:11:29.498" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:10:59.599" grpc.time_ms=13.806 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.729273] sidekiq-cluster[3494]: {"severity":"WARN","time":"2026-10-04T19:10:59.650Z","class":"CreatePipelineWorker","correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"760c67a24da4b813ae4e4eb76ca01815d143bd41","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"760c67a24da4b813ae4e4eb76ca01815d143bd41","ref":"refs/heads/master"},"retry":0} gitlab # [ 424.738756] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.660Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2c41a85cf5769d4832007edd","created_at":"2026-10-04T19:10:57.167Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e0beb0d44f5447fcbab8839b6a369c169a22acadc55291301fde3ecfae4c5ea6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.174Z","job_size_bytes":467,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2c41a85cf5769d4832007edd: done: 0.944015 sec","job_status":"done","queue_duration_s":1.542081,"scheduling_latency_s":1.542105,"gitaly_calls":7,"gitaly_duration_s":0.067794,"redis_calls":51,"redis_duration_s":0.009295,"redis_read_bytes":169,"redis_write_bytes":4167,"redis_cache_calls":4,"redis_cache_duration_s":0.000841,"redis_cache_read_bytes":10,"redis_cache_write_bytes":342,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.003802,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002485,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001019,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.03,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.823457,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033152,"duration_s":0.944015,"completed_at":"2026-10-04T19:10:59.660Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.058076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.792804] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.662Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"0eebfe6e-4366-48cd-a956-c62b83de08a5\"}"],"class":"WebHookWorker","jid":"97d31239141ab0cac36e20c8","created_at":"2026-10-04T19:10:57.322Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d3566a038d4fbd694b843f05aff22968e85713a2293ebd795d781614c666a6a6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.328Z","job_size_bytes":1751,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-97d31239141ab0cac36e20c8: start","job_status":"start","queue_duration_s":2.334178,"scheduling_latency_s":2.334205} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 424.900993] nixbot[3615]: INFO: ::ffff:127.0.0.1:43222 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 424.928570] nixbot[3615]: INFO: ::ffff:127.0.0.1:43236 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.04 seconds) gitlab # [ 424.932484] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.849Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"0eebfe6e-4366-48cd-a956-c62b83de08a5\"}"],"class":"WebHookWorker","jid":"97d31239141ab0cac36e20c8","created_at":"2026-10-04T19:10:57.322Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-34327e8b23bc4e5a","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d3566a038d4fbd694b843f05aff22968e85713a2293ebd795d781614c666a6a6","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.328Z","job_size_bytes":1751,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-97d31239141ab0cac36e20c8: done: 0.187584 sec","job_status":"done","queue_duration_s":2.334178,"scheduling_latency_s":2.334205,"redis_calls":19,"redis_duration_s":0.006497,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001662,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000549,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.0034,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06644425999996884,"cpu_s":0.104244,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000687,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.187584,"completed_at":"2026-10-04T19:10:59.849Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.009755] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.852Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"83d9366063e2927cb9ac95c1","created_at":"2026-10-04T19:10:57.868Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.873Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-83d9366063e2927cb9ac95c1: start","job_status":"start","queue_duration_s":1.978761,"scheduling_latency_s":1.978791} gitlab # [ 425.030536] gitaly[923]: time="2026-10-04T19:10:59.940Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=5 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M445778B9H7X0N37R0AK83AR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:10:59.928" grpc.time_ms=12.663 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 425.045886] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:10:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 89 gitlab # [ 425.048248] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.959Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"83d9366063e2927cb9ac95c1","created_at":"2026-10-04T19:10:57.868Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:57.873Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-83d9366063e2927cb9ac95c1: done: 0.106527 sec","job_status":"done","queue_duration_s":1.978761,"scheduling_latency_s":1.978791,"redis_calls":10,"redis_duration_s":0.0019500000000000001,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000407,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025334,"duration_s":0.106527,"completed_at":"2026-10-04T19:10:59.959Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018642,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.095501] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.960Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e918341c6efd95075156b78","created_at":"2026-10-04T19:10:57.995Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.001Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e918341c6efd95075156b78: start","job_status":"start","queue_duration_s":1.95938,"scheduling_latency_s":1.959404} gitlab # [ 425.113020] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.998Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"9e918341c6efd95075156b78","created_at":"2026-10-04T19:10:57.995Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.001Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-9e918341c6efd95075156b78: done: 0.037558 sec","job_status":"done","queue_duration_s":1.95938,"scheduling_latency_s":1.959404,"redis_calls":10,"redis_duration_s":0.001516,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000288,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000327,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00023914,"duration_s":0.037558,"completed_at":"2026-10-04T19:10:59.998Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005192,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.159122] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:10:59.999Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"34057779df4b3f402384e4e0","created_at":"2026-10-04T19:10:58.017Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.023Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-34057779df4b3f402384e4e0: start","job_status":"start","queue_duration_s":1.976752,"scheduling_latency_s":1.976772} gitlab # [ 425.176384] gitlab-workhorse[822]: time="2026-10-04T19:11:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 425.178344] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 71 gitlab # [ 425.181830] gitlab-workhorse[822]: time="2026-10-04T19:11:00Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=336 gitlab # [ 425.183907] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 336 "" "git/2.55.0" 79 gitlab # [ 425.186284] gitaly[923]: time="2026-10-04T19:11:00.013Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M44577B76BVG069EDRZN8A8X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.003" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 425.197519] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"34057779df4b3f402384e4e0","created_at":"2026-10-04T19:10:58.017Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.023Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-34057779df4b3f402384e4e0: done: 0.039012 sec","job_status":"done","queue_duration_s":1.976752,"scheduling_latency_s":1.976772,"redis_calls":10,"redis_duration_s":0.001905,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001042,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000414,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029536,"duration_s":0.039012,"completed_at":"2026-10-04T19:11:00.038Z","load_balancing_strategy":"primary","db_duration_s":0.005294,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.242866] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a8aa30019f4de42636e14e45","created_at":"2026-10-04T19:10:58.039Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.044Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a8aa30019f4de42636e14e45: start","job_status":"start","queue_duration_s":1.996003,"scheduling_latency_s":1.996024} gitlab # [ 425.259064] gitaly[923]: time="2026-10-04T19:11:00.013Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M44577B76BVG069EDRZN8A8X grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.003" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 425.272896] nixbot[3615]: {"ts": "2026-10-04T19:11:00.142Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 425.275392] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.079Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a8aa30019f4de42636e14e45","created_at":"2026-10-04T19:10:58.039Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.044Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a8aa30019f4de42636e14e45: done: 0.039712 sec","job_status":"done","queue_duration_s":1.996003,"scheduling_latency_s":1.996024,"redis_calls":13,"redis_duration_s":0.003255,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000341,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001049,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001246,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000619,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02636,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024854,"duration_s":0.039712,"completed_at":"2026-10-04T19:11:00.079Z","load_balancing_strategy":"primary","db_duration_s":0.005217,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.322577] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.081Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8cf7ea470a2be08fa4eb90f6","created_at":"2026-10-04T19:10:58.044Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.049Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8cf7ea470a2be08fa4eb90f6: start","job_status":"start","queue_duration_s":2.031636,"scheduling_latency_s":2.031654} gitlab # [ 425.339671] gitaly[923]: time="2026-10-04T19:11:00.015Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M44577B76BVG069EDRZN8A8X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.003" grpc.time_ms=10.194 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 425.354883] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.124Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"8cf7ea470a2be08fa4eb90f6","created_at":"2026-10-04T19:10:58.044Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.049Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-8cf7ea470a2be08fa4eb90f6: done: 0.043102 sec","job_status":"done","queue_duration_s":2.031636,"scheduling_latency_s":2.031654,"redis_calls":10,"redis_duration_s":0.005338,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002034,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002905,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00072312,"duration_s":0.043102,"completed_at":"2026-10-04T19:11:00.124Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006344,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.402205] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.126Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d799a64e5da7c5d60e9940f1","created_at":"2026-10-04T19:10:58.050Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.056Z","job_size_bytes":19,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d799a64e5da7c5d60e9940f1: start","job_status":"start","queue_duration_s":2.070157,"scheduling_latency_s":2.070181} gitlab # [ 425.419198] gitaly[923]: time="2026-10-04T19:11:00.094Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M44577DGP7EHEYR0HQ2C7ES6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.075" pid=923 remote_ip="2001:db8:1::1" request_sha=4e8c81c48bf1e7ced59553d2ebfc151019a47b32 response_bytes=336 span.kind=server system=grpc gitlab # [ 425.430000] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.172Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"d799a64e5da7c5d60e9940f1","created_at":"2026-10-04T19:10:58.050Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.056Z","job_size_bytes":19,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-d799a64e5da7c5d60e9940f1: done: 0.046355 sec","job_status":"done","queue_duration_s":2.070157,"scheduling_latency_s":2.070181,"redis_calls":11,"redis_duration_s":0.003942,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000483,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002997,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031992,"duration_s":0.046355,"completed_at":"2026-10-04T19:11:00.172Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005249,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.475748] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.174Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ba1d5e5a02a7d9d4aa0ad5ff","created_at":"2026-10-04T19:10:58.059Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.063Z","job_size_bytes":66,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ba1d5e5a02a7d9d4aa0ad5ff: start","job_status":"start","queue_duration_s":2.110653,"scheduling_latency_s":2.110678} gitlab # [ 425.493943] gitaly[923]: time="2026-10-04T19:11:00.094Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577DGP7EHEYR0HQ2C7ES6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.075" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 425.508335] nixbot[3615]: {"ts": "2026-10-04T19:11:00.361Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 425.511208] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.215Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"ba1d5e5a02a7d9d4aa0ad5ff","created_at":"2026-10-04T19:10:58.059Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.063Z","job_size_bytes":66,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-ba1d5e5a02a7d9d4aa0ad5ff: done: 0.040788 sec","job_status":"done","queue_duration_s":2.110653,"scheduling_latency_s":2.110678,"redis_calls":10,"redis_duration_s":0.001885,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000351,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000378,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027926,"duration_s":0.040788,"completed_at":"2026-10-04T19:11:00.215Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005211,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.558855] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.217Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ab768d160c599091971c0942","created_at":"2026-10-04T19:10:58.064Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.069Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ab768d160c599091971c0942: start","job_status":"start","queue_duration_s":2.147173,"scheduling_latency_s":2.147195} gitlab # [ 425.575832] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 204 gitlab # [ 425.578619] gitaly[923]: time="2026-10-04T19:11:00.095Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1128 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577DGP7EHEYR0HQ2C7ES6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.075" grpc.time_ms=19.087 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 425.595040] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.269Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ab768d160c599091971c0942","created_at":"2026-10-04T19:10:58.064Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.069Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ab768d160c599091971c0942: done: 0.052439 sec","job_status":"done","queue_duration_s":2.147173,"scheduling_latency_s":2.147195,"redis_calls":10,"redis_duration_s":0.002213,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000347,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00127,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000596,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038928,"duration_s":0.052439,"completed_at":"2026-10-04T19:11:00.269Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006214,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.640742] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.271Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c01705ee77cfa494a5c445eb","created_at":"2026-10-04T19:10:58.077Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.081Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c01705ee77cfa494a5c445eb: start","job_status":"start","queue_duration_s":2.189866,"scheduling_latency_s":2.189887} gitlab # [ 425.657721] gitaly[923]: time="2026-10-04T19:11:00.230Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:30.228" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.229" grpc.time_ms=0.805 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.671131] nixbot[3615]: {"ts": "2026-10-04T19:11:00.522Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab # [ 425.674173] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.319Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"c01705ee77cfa494a5c445eb","created_at":"2026-10-04T19:10:58.077Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.081Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-c01705ee77cfa494a5c445eb: done: 0.048865 sec","job_status":"done","queue_duration_s":2.189866,"scheduling_latency_s":2.189887,"redis_calls":10,"redis_duration_s":0.004148,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000426,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000412,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00152292,"duration_s":0.048865,"completed_at":"2026-10-04T19:11:00.319Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.720210] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.322Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"623216d03f625c65609076a5","created_at":"2026-10-04T19:10:58.081Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.086Z","job_size_bytes":6,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-623216d03f625c65609076a5: start","job_status":"start","queue_duration_s":2.236308,"scheduling_latency_s":2.236331} gitlab # [ 425.736680] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 156 gitlab # [ 425.739932] gitaly[923]: time="2026-10-04T19:11:00.301Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:10.172" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:00.292" grpc.time_ms=8.392 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.754576] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.354Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"623216d03f625c65609076a5","created_at":"2026-10-04T19:10:58.081Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.086Z","job_size_bytes":6,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-623216d03f625c65609076a5: done: 0.031932 sec","job_status":"done","queue_duration_s":2.236308,"scheduling_latency_s":2.236331,"redis_calls":10,"redis_duration_s":0.004108,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000796,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002443,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018278,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00088182,"duration_s":0.031932,"completed_at":"2026-10-04T19:11:00.354Z","load_balancing_strategy":"primary","db_duration_s":0.002707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.801555] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.357Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"9ab1093366596974cafa9b3d","created_at":"2026-10-04T19:10:58.086Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.090Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9ab1093366596974cafa9b3d: start","job_status":"start","queue_duration_s":2.266407,"scheduling_latency_s":2.266429} gitlab # [ 425.817599] gitaly[923]: time="2026-10-04T19:11:00.420Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577RDDCSNDBP0Y3GG1PG3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:30.419" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.419" grpc.time_ms=0.66 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.830879] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.399Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"9ab1093366596974cafa9b3d","created_at":"2026-10-04T19:10:58.086Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.090Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-9ab1093366596974cafa9b3d: done: 0.042179 sec","job_status":"done","queue_duration_s":2.266407,"scheduling_latency_s":2.266429,"redis_calls":10,"redis_duration_s":0.002985,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000879,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001754,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040352,"duration_s":0.042179,"completed_at":"2026-10-04T19:11:00.399Z","load_balancing_strategy":"primary","db_duration_s":0.009049,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.875964] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.402Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"26b57cdd9bc42f822861bda6","created_at":"2026-10-04T19:10:58.096Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.101Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-26b57cdd9bc42f822861bda6: start","job_status":"start","queue_duration_s":2.301235,"scheduling_latency_s":2.301258} gitlab # [ 425.893078] gitaly[923]: time="2026-10-04T19:11:00.467Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44577RDDCSNDBP0Y3GG1PG3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:10.027" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:00.458" grpc.time_ms=8.862 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.907002] nixbot[3615]: {"ts": "2026-10-04T19:11:00.802Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 \"HTTP/1.1 201 Created\""} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 425.910522] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.628Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"26b57cdd9bc42f822861bda6","created_at":"2026-10-04T19:10:58.096Z","trace_propagation_headers":{"sentry-trace":"08539e7bf056405daff8be0b539c83f9-042b0b7d82f0419c","baggage":"sentry-trace_id=08539e7bf056405daff8be0b539c83f9,sentry-release=Unknown"},"correlation_id":"01M4454MGB7MZ3KG17SRDMED0Q","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.101Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-26b57cdd9bc42f822861bda6: done: 0.226169 sec","job_status":"done","queue_duration_s":2.301235,"scheduling_latency_s":2.301258,"redis_calls":14,"redis_duration_s":0.0059889999999999995,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00392,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001613,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.060205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00260768,"duration_s":0.226169,"completed_at":"2026-10-04T19:11:00.628Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016505,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.955455] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.630Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2d43e4a8a9313eb9307cdc93","created_at":"2026-10-04T19:10:58.259Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"c8a35ab3c63ff819b37faa98","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.263Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2d43e4a8a9313eb9307cdc93: start","job_status":"start","queue_duration_s":2.367433,"scheduling_latency_s":2.367461} gitlab # [ 425.973399] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/760c67a24da4b813ae4e4eb76ca01815d143bd41 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 272 gitlab # [ 425.976032] gitaly[923]: time="2026-10-04T19:11:00.614Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577XG5EFQP1JNFMEPK7N6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:30.312" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.613" grpc.time_ms=0.762 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.989100] gitaly[923]: time="2026-10-04T19:11:00.661Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX diskcache=f73a809b-1960-46fe-9ebe-c8909840f4ce grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:11:00.653" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.653" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.07 seconds) gitlab # [ 426.000732] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.688Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"2d43e4a8a9313eb9307cdc93","created_at":"2026-10-04T19:10:58.259Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"c8a35ab3c63ff819b37faa98","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.263Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-2d43e4a8a9313eb9307cdc93: done: 0.057952 sec","job_status":"done","queue_duration_s":2.367433,"scheduling_latency_s":2.367461,"gitaly_calls":1,"gitaly_duration_s":0.010455,"redis_calls":12,"redis_duration_s":0.002558,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000645,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03382024,"duration_s":0.057952,"completed_at":"2026-10-04T19:11:00.688Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013259,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.048614] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:00.690Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e4c1917ecb4ceb84b0cd29ac","created_at":"2026-10-04T19:10:58.380Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:79fb19c01c802b5e6e1f60509b0ebe659ba1a9c267072f81355d808a2c64064c","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.383Z","job_size_bytes":467,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e4c1917ecb4ceb84b0cd29ac: start","job_status":"start","queue_duration_s":2.306906,"scheduling_latency_s":2.306936} gitlab # [ 426.067024] gitaly[923]: time="2026-10-04T19:11:00.661Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-05T01:11:00.653" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.653" grpc.time_ms=7.95 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.081313] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:00 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 73 gitlab # [ 426.083721] nixbot[3615]: INFO: ::ffff:127.0.0.1:43238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab # [ 426.085854] gitaly[923]: time="2026-10-04T19:11:00.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44577XG5EFQP1JNFMEPK7N6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:10.117" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:00.707" grpc.time_ms=11.406 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.099733] gitaly[923]: time="2026-10-04T19:11:00.722Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:30.621" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.721" grpc.time_ms=0.597 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.112880] gitlab-workhorse[822]: time="2026-10-04T19:11:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 426.114750] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 99 gitlab # [ 426.116717] gitaly[923]: time="2026-10-04T19:11:00.735Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:11:10.004" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:00.725" grpc.time_ms=10.119 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.130184] gitaly[923]: time="2026-10-04T19:11:00.780Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:11:30.679" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.779" grpc.time_ms=0.849 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.143451] gitaly[923]: time="2026-10-04T19:11:00.789Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=76 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-04T19:11:30.684" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:00.785" grpc.time_ms=3.975 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.160171] gitaly[923]: time="2026-10-04T19:11:00.839Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:10.108" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:00.829" grpc.time_ms=9.756 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.173759] gitaly[923]: time="2026-10-04T19:11:00.906Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=577 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=11 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M445787129XFEG2XPHNV0J2S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.888" grpc.time_ms=17.653 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.187511] gitaly[923]: time="2026-10-04T19:11:01.007Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M445789DY3KNP4NV0F4A6F4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.995" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 426.198305] gitaly[923]: time="2026-10-04T19:11:01.007Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M445789DY3KNP4NV0F4A6F4A grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.995" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.213852] gitaly[923]: time="2026-10-04T19:11:01.007Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M445789DY3KNP4NV0F4A6F4A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:00.995" grpc.time_ms=12.01 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.442509] gitaly[923]: time="2026-10-04T19:11:01.364Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-04T19:11:31.248" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:01.348" grpc.time_ms=15.51 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.467240] gitaly[923]: time="2026-10-04T19:11:01.378Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M445715A32X6Y3Q1M1V3H3VX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-04T19:11:31.267" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:01.367" grpc.time_ms=10.295 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.488209] sidekiq-cluster[3494]: {"severity":"WARN","time":"2026-10-04T19:11:01.394Z","class":"CreatePipelineWorker","correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"760c67a24da4b813ae4e4eb76ca01815d143bd41","after":"459fc935a5411646cc3e2e58cd0153ee2c26ddcf","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"459fc935a5411646cc3e2e58cd0153ee2c26ddcf","ref":"refs/heads/master"},"retry":0} gitlab # [ 426.498125] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.401Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"e4c1917ecb4ceb84b0cd29ac","created_at":"2026-10-04T19:10:58.380Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:79fb19c01c802b5e6e1f60509b0ebe659ba1a9c267072f81355d808a2c64064c","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.383Z","job_size_bytes":467,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-e4c1917ecb4ceb84b0cd29ac: done: 0.711105 sec","job_status":"done","queue_duration_s":2.306906,"scheduling_latency_s":2.306936,"gitaly_calls":7,"gitaly_duration_s":0.069542,"redis_calls":33,"redis_duration_s":0.0051389999999999995,"redis_read_bytes":185,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000537,"redis_cache_read_bytes":44,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002665,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.620574,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.711105,"completed_at":"2026-10-04T19:11:01.401Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022008,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.554137] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.403Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"3692d255-cd72-47bf-b99b-2eb037b9e8a0\"}"],"class":"WebHookWorker","jid":"5a00a3ad9b6c02988bff38e8","created_at":"2026-10-04T19:10:58.400Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0445579d1a047970192a93532020df3292a11819b87f9893a911d499b19a976a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.403Z","job_size_bytes":1749,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5a00a3ad9b6c02988bff38e8: start","job_status":"start","queue_duration_s":3.000386,"scheduling_latency_s":3.000417} gitlab # [ 426.573119] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.456Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"3692d255-cd72-47bf-b99b-2eb037b9e8a0\"}"],"class":"WebHookWorker","jid":"5a00a3ad9b6c02988bff38e8","created_at":"2026-10-04T19:10:58.400Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-d30b5c9a4b944d65","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0445579d1a047970192a93532020df3292a11819b87f9893a911d499b19a976a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.403Z","job_size_bytes":1749,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-5a00a3ad9b6c02988bff38e8: done: 0.052895 sec","job_status":"done","queue_duration_s":3.000386,"scheduling_latency_s":3.000417,"redis_calls":14,"redis_duration_s":0.004935,"redis_read_bytes":25,"redis_write_bytes":4685,"redis_queues_calls":2,"redis_queues_duration_s":0.000361,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003401,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001173,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.00700963999997839,"cpu_s":0.033274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.052895,"completed_at":"2026-10-04T19:11:01.456Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00223,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.622913] nixbot[3615]: INFO: ::ffff:127.0.0.1:43250 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 426.624220] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 65 gitlab # [ 426.626360] gitaly[923]: time="2026-10-04T19:11:01.536Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44578TZ979Q9J7EZ6E75FJH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.526" grpc.time_ms=9.836 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.640255] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.458Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"12cb8dfb5244c6fac177c729","created_at":"2026-10-04T19:10:58.535Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-f0ed6f0e58be4be6","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.541Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-12cb8dfb5244c6fac177c729: start","job_status":"start","queue_duration_s":2.916751,"scheduling_latency_s":2.916777} gitlab # [ 426.656649] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.531Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"12cb8dfb5244c6fac177c729","created_at":"2026-10-04T19:10:58.535Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-f0ed6f0e58be4be6","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:58.541Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-12cb8dfb5244c6fac177c729: done: 0.072923 sec","job_status":"done","queue_duration_s":2.916751,"scheduling_latency_s":2.916777,"redis_calls":19,"redis_duration_s":0.009658,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001832,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000564,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.003452,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00381,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011267039999999999,"duration_s":0.072923,"completed_at":"2026-10-04T19:11:01.531Z","load_balancing_strategy":"primary","db_duration_s":0.007494,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.703660] gitaly[923]: time="2026-10-04T19:11:01.602Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M44578X2K6VXX7ZG194K6KQ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.592" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 426.714364] gitlab-workhorse[822]: time="2026-10-04T19:11:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 426.716297] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 64 gitlab # [ 426.718243] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.533Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","97d31239141ab0cac36e20c8"],"class":"WebHooks::LogExecutionWorker","jid":"30cc47f6688237550a3b7a85","created_at":"2026-10-04T19:10:59.826Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-6abae342e8694945","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:de68b6249c7b79a4039f469f3876325095a938e05992b5cd2a0ed670b2ebd31a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:59.837Z","job_size_bytes":2593,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-30cc47f6688237550a3b7a85: start","job_status":"start","queue_duration_s":1.696015,"scheduling_latency_s":1.696047} gitlab # [ 426.736582] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.595Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","97d31239141ab0cac36e20c8"],"class":"WebHooks::LogExecutionWorker","jid":"30cc47f6688237550a3b7a85","created_at":"2026-10-04T19:10:59.826Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-6abae342e8694945","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:de68b6249c7b79a4039f469f3876325095a938e05992b5cd2a0ed670b2ebd31a","size_limiter":"validated","enqueued_at":"2026-10-04T19:10:59.837Z","job_size_bytes":2593,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-30cc47f6688237550a3b7a85: done: 0.061977 sec","job_status":"done","queue_duration_s":1.696015,"scheduling_latency_s":1.696047,"redis_calls":10,"redis_duration_s":0.003467,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000548,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042966,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027156,"duration_s":0.061977,"completed_at":"2026-10-04T19:11:01.595Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008667,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 426.783192] gitaly[923]: time="2026-10-04T19:11:01.603Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M44578X2K6VXX7ZG194K6KQ0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.592" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.797577] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.597Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"81c3d7ef5d3f0154fe6ec01b","created_at":"2026-10-04T19:11:00.337Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-276b095b72e8464d","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:00.342Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-81c3d7ef5d3f0154fe6ec01b: start","job_status":"start","queue_duration_s":1.255141,"scheduling_latency_s":1.255167} gitlab # [ 426.814137] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:01 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 67 gitlab # [ 426.816357] gitaly[923]: time="2026-10-04T19:11:01.603Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M44578X2K6VXX7ZG194K6KQ0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.592" grpc.time_ms=10.937 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.832444] gitaly[923]: time="2026-10-04T19:11:01.706Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M4457907CC7219ZPBB4F10A5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.698" grpc.time_ms=8.486 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.845818] gitaly[923]: time="2026-10-04T19:11:01.737Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:11:01.737" grpc.time_ms=0.49 pid=923 span.kind=server system=grpc gitlab # [ 426.855892] gitaly[923]: time="2026-10-04T19:11:01.773Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:11:01.773" grpc.time_ms=0.544 pid=923 span.kind=server system=grpc gitlab # [ 426.873695] gitaly[923]: time="2026-10-04T19:11:01.795Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M445792D495DQ960JNSXM9FA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.771" pid=923 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=221 span.kind=server system=grpc gitlab # [ 426.885982] gitaly[923]: time="2026-10-04T19:11:01.795Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445792D495DQ960JNSXM9FA grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.771" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.900690] gitlab-workhorse[822]: time="2026-10-04T19:11:01Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=221 gitlab # [ 426.902587] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:01 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 221 "" "git/2.55.0" 87 gitlab # [ 426.904573] gitaly[923]: time="2026-10-04T19:11:01.796Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1104 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M445792D495DQ960JNSXM9FA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:01.771" grpc.time_ms=24.526 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 426.920182] gitaly[923]: time="2026-10-04T19:11:01.802Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:11:01.802" grpc.time_ms=0.418 pid=923 span.kind=server system=grpc gitlab # [ 426.930570] gitaly[923]: time="2026-10-04T19:11:01.813Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ diskcache=10713b6f-9471-4e1d-8bda-3fd4012da734 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:11:31.603" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:01.703" pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.940833] gitaly[923]: time="2026-10-04T19:11:01.813Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=101 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7441 command.oublock=0 command.real_time_ms=105 command.spawn_token_fork_ms=1 command.system_time_ms=67 command.user_time_ms=34 component=gitaly.UnaryServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-04T19:11:31.603" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:01.703" grpc.time_ms=110.154 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.960816] gitaly[923]: time="2026-10-04T19:11:01.882Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:11:31.781" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:01.881" grpc.time_ms=0.788 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.977424] gitaly[923]: time="2026-10-04T19:11:01.899Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:31.598" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:01.898" grpc.time_ms=0.683 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf' gitlab # [ 426.999493] gitaly[923]: time="2026-10-04T19:11:01.921Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=592 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-04T19:11:31.612" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:01.912" grpc.time_ms=8.328 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.013668] gitaly[923]: time="2026-10-04T19:11:01.924Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44577HTB5CN5PG1WZ1J69JZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:31.823" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:01.923" grpc.time_ms=0.535 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.027586] gitaly[923]: time="2026-10-04T19:11:01.939Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:11.020" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:01.930" grpc.time_ms=9.024 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.042023] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.937Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"81c3d7ef5d3f0154fe6ec01b","created_at":"2026-10-04T19:11:00.337Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-276b095b72e8464d","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:00.342Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3494,"job_status":"rescheduled","message":"PipelineProcessWorker JID-81c3d7ef5d3f0154fe6ec01b: rescheduled"} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf', in 0.08 seconds) (finished: subtest: push delivers the webhook, in 87.47 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 427.057830] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.944Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"81c3d7ef5d3f0154fe6ec01b","created_at":"2026-10-04T19:11:00.337Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-276b095b72e8464d","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:00.342Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-81c3d7ef5d3f0154fe6ec01b: done: 0.347048 sec","job_status":"done","queue_duration_s":1.255141,"scheduling_latency_s":1.255167,"gitaly_calls":3,"gitaly_duration_s":0.118402,"redis_calls":67,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.018738,"redis_read_bytes":149,"redis_write_bytes":18098,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000648,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002069,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00416,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12312,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.004682,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000623,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.003599,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002309,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.047,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.009,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.168744,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.29231058,"duration_s":0.347048,"completed_at":"2026-10-04T19:11:01.944Z","load_balancing_strategy":"primary","db_duration_s":0.050243,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.113371] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.945Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5a00a3ad9b6c02988bff38e8"],"class":"WebHooks::LogExecutionWorker","jid":"80db0ba2850943b6198cde27","created_at":"2026-10-04T19:11:01.442Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-3c40af975b38493b","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:db87c26e5922340d2c0520ba3260743b32928166d112b0e4c361ddddcd86fc31","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.445Z","job_size_bytes":2592,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-80db0ba2850943b6198cde27: start","job_status":"start","queue_duration_s":0.50067,"scheduling_latency_s":0.500697} gitlab # [ 427.130894] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.979Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","5a00a3ad9b6c02988bff38e8"],"class":"WebHooks::LogExecutionWorker","jid":"80db0ba2850943b6198cde27","created_at":"2026-10-04T19:11:01.442Z","trace_propagation_headers":{"sentry-trace":"cba20785b3204061970779dfd6376c50-3c40af975b38493b","baggage":"sentry-trace_id=cba20785b3204061970779dfd6376c50,sentry-release=Unknown"},"correlation_id":"01M445715A32X6Y3Q1M1V3H3VX","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:db87c26e5922340d2c0520ba3260743b32928166d112b0e4c361ddddcd86fc31","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.445Z","job_size_bytes":2592,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-80db0ba2850943b6198cde27: done: 0.033693 sec","job_status":"done","queue_duration_s":0.50067,"scheduling_latency_s":0.500697,"redis_calls":7,"redis_duration_s":0.004994,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002784,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00221,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033693,"completed_at":"2026-10-04T19:11:01.979Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005642,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.177466] nixbot[3615]: INFO: ::ffff:127.0.0.1:43252 - "GET /api/repos/gitlab/root/test-flake/builds?commit=459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 200 OK gitlab # [ 427.179800] gitaly[923]: time="2026-10-04T19:11:01.985Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-04T19:11:11.075" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:11:01.985" grpc.time_ms=0.334 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.191496] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:01.981Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f0e033f05843527171003fd8","created_at":"2026-10-04T19:11:01.506Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-f65ae9203aa34b4b","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.513Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f0e033f05843527171003fd8: start","job_status":"start","queue_duration_s":0.467099,"scheduling_latency_s":0.467125} gitlab # [ 427.209471] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.082Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"f0e033f05843527171003fd8","created_at":"2026-10-04T19:11:01.506Z","trace_propagation_headers":{"sentry-trace":"a10cd71545da4134bb47fcb2b3c2a893-f65ae9203aa34b4b","baggage":"sentry-trace_id=a10cd71545da4134bb47fcb2b3c2a893,sentry-release=Unknown"},"correlation_id":"01M4454EYVSDPR9PFBYBH0VGP9","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.513Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-f0e033f05843527171003fd8: done: 0.101292 sec","job_status":"done","queue_duration_s":0.467099,"scheduling_latency_s":0.467125,"redis_calls":11,"redis_duration_s":0.007693,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000721,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00651,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.072441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00122622,"duration_s":0.101292,"completed_at":"2026-10-04T19:11:02.082Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.18 seconds) gitlab # [ 427.258056] gitaly[923]: time="2026-10-04T19:11:02.001Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=569 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-04T19:11:11.082" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:01.992" grpc.time_ms=9.165 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.271994] gitaly[923]: time="2026-10-04T19:11:02.043Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:11:02.041" grpc.time_ms=0.915 pid=923 span.kind=server system=grpc gitlab # [ 427.282338] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:02 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 2 "" "curl/8.21.0" 109 gitlab # [ 427.285111] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.083Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3cca1cc045efe0505946436e","created_at":"2026-10-04T19:11:01.690Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.693Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3cca1cc045efe0505946436e: start","job_status":"start","queue_duration_s":0.390253,"scheduling_latency_s":0.390275} gitlab # [ 427.302317] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.135Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"3cca1cc045efe0505946436e","created_at":"2026-10-04T19:11:01.690Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.693Z","job_size_bytes":23,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-3cca1cc045efe0505946436e: done: 0.051254 sec","job_status":"done","queue_duration_s":0.390253,"scheduling_latency_s":0.390275,"redis_calls":7,"redis_duration_s":0.002778,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002379,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051254,"completed_at":"2026-10-04T19:11:02.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013775,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.346742] gitaly[923]: time="2026-10-04T19:11:02.078Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-04T19:11:02.076" grpc.time_ms=0.596 pid=923 span.kind=server system=grpc gitlab # [ 427.356901] gitaly[923]: time="2026-10-04T19:11:02.088Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC diskcache=03873694-76a9-4444-9f05-c93a3360c40f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:11:12.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:11:02.004" pid=923 ref=refs/keep-around/459fc935a5411646cc3e2e58cd0153ee2c26ddcf remote_ip="2001:db8:1::1" revision=459fc935a5411646cc3e2e58cd0153ee2c26ddcf span.kind=server system=grpc username=root gitlab # [ 427.368772] nixbot[3615]: {"ts": "2026-10-04T19:11:02.206Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf \"HTTP/1.1 201 Created\""} gitlab # [ 427.371543] nixbot[3615]: {"ts": "2026-10-04T19:11:02.214Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-459fc935a541-5"} gitlab # [ 427.374049] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:02 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 375 gitlab # [ 427.376697] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.136Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8f3b244f007cb783ff4e88c8","created_at":"2026-10-04T19:11:01.820Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.824Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8f3b244f007cb783ff4e88c8: start","job_status":"start","queue_duration_s":0.312168,"scheduling_latency_s":0.312188} gitlab # [ 427.393995] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.167Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8f3b244f007cb783ff4e88c8","created_at":"2026-10-04T19:11:01.820Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.824Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8f3b244f007cb783ff4e88c8: done: 0.0317 sec","job_status":"done","queue_duration_s":0.312168,"scheduling_latency_s":0.312188,"redis_calls":7,"redis_duration_s":0.001056,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000303,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020933,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0317,"completed_at":"2026-10-04T19:11:02.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.439685] gitaly[923]: time="2026-10-04T19:11:02.088Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=62 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5187 command.oublock=8 command.real_time_ms=76 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-04T19:11:12.004" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-04T19:11:02.004" grpc.time_ms=83.679 pid=923 ref=refs/keep-around/459fc935a5411646cc3e2e58cd0153ee2c26ddcf remote_ip="2001:db8:1::1" revision=459fc935a5411646cc3e2e58cd0153ee2c26ddcf span.kind=server system=grpc username=root gitlab # [ 427.455070] nixbot[3615]: {"ts": "2026-10-04T19:11:02.352Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-459fc935a541-5", "argv0": "bwrap"} gitlab # [ 427.457575] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.170Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c4ea522e8714c62faf349597","created_at":"2026-10-04T19:11:01.834Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.837Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c4ea522e8714c62faf349597: start","job_status":"start","queue_duration_s":0.332967,"scheduling_latency_s":0.332986} gitlab # [ 427.474005] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.204Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c4ea522e8714c62faf349597","created_at":"2026-10-04T19:11:01.834Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.837Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c4ea522e8714c62faf349597: done: 0.033528 sec","job_status":"done","queue_duration_s":0.332967,"scheduling_latency_s":0.332986,"redis_calls":7,"redis_duration_s":0.0028510000000000002,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021782,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033528,"completed_at":"2026-10-04T19:11:02.203Z","load_balancing_strategy":"primary","db_duration_s":0.004539,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.518030] gitaly[923]: time="2026-10-04T19:11:02.094Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:32.093" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:02.093" grpc.time_ms=0.389 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.539010] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.205Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a65ca0b87a60fc9c60ef8469","created_at":"2026-10-04T19:11:01.850Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.853Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a65ca0b87a60fc9c60ef8469: start","job_status":"start","queue_duration_s":0.352232,"scheduling_latency_s":0.352252} gitlab # [ 427.563944] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.237Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"a65ca0b87a60fc9c60ef8469","created_at":"2026-10-04T19:11:01.850Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.853Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-a65ca0b87a60fc9c60ef8469: done: 0.032077 sec","job_status":"done","queue_duration_s":0.352232,"scheduling_latency_s":0.352252,"redis_calls":10,"redis_duration_s":0.001717,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000455,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000365,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032077,"completed_at":"2026-10-04T19:11:02.237Z","load_balancing_strategy":"primary","db_duration_s":0.00431,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.622752] gitaly[923]: time="2026-10-04T19:11:02.133Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44579D52CA0ABEYYQ5X0J6Y grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:32.132" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:02.132" grpc.time_ms=0.643 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.636988] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.238Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fab46a2b686867a5abc89742","created_at":"2026-10-04T19:11:01.853Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.855Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fab46a2b686867a5abc89742: start","job_status":"start","queue_duration_s":0.383023,"scheduling_latency_s":0.383044} gitlab # [ 427.655524] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.265Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"fab46a2b686867a5abc89742","created_at":"2026-10-04T19:11:01.853Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.855Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-fab46a2b686867a5abc89742: done: 0.02678 sec","job_status":"done","queue_duration_s":0.383023,"scheduling_latency_s":0.383044,"redis_calls":7,"redis_duration_s":0.0015149999999999999,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001158,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018682,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02678,"completed_at":"2026-10-04T19:11:02.265Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.701735] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.266Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a239c4ce5235aba0e090d943","created_at":"2026-10-04T19:11:01.856Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.859Z","job_size_bytes":19,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a239c4ce5235aba0e090d943: start","job_status":"start","queue_duration_s":0.407858,"scheduling_latency_s":0.407876} gitlab # [ 427.718907] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"a239c4ce5235aba0e090d943","created_at":"2026-10-04T19:11:01.856Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.859Z","job_size_bytes":19,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-a239c4ce5235aba0e090d943: done: 0.024299 sec","job_status":"done","queue_duration_s":0.407858,"scheduling_latency_s":0.407876,"redis_calls":8,"redis_duration_s":0.001147,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000887,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024299,"completed_at":"2026-10-04T19:11:02.291Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003057,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.762641] gitaly[923]: time="2026-10-04T19:11:02.597Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-04T19:11:32.496" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:02.596" grpc.time_ms=0.917 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.775884] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.292Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b209bdf265bce5dc3d0afce6","created_at":"2026-10-04T19:11:01.860Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.862Z","job_size_bytes":66,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b209bdf265bce5dc3d0afce6: start","job_status":"start","queue_duration_s":0.429986,"scheduling_latency_s":0.430025} gitlab # [ 427.793774] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.317Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b209bdf265bce5dc3d0afce6","created_at":"2026-10-04T19:11:01.860Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.862Z","job_size_bytes":66,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b209bdf265bce5dc3d0afce6: done: 0.024413 sec","job_status":"done","queue_duration_s":0.429986,"scheduling_latency_s":0.430025,"redis_calls":7,"redis_duration_s":0.001132,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000824,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017904,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024413,"completed_at":"2026-10-04T19:11:02.317Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002197,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.838913] nix-daemon[3777]: accepted connection from pid 4835, user nixbot gitlab # [ 427.839911] gitaly[923]: time="2026-10-04T19:11:02.649Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M44579694NVANNPWYRQ918KC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:32.548" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:02.648" grpc.time_ms=0.702 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.853301] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.318Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b39ddae2ce675522eacb8fad","created_at":"2026-10-04T19:11:01.863Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.865Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b39ddae2ce675522eacb8fad: start","job_status":"start","queue_duration_s":0.453027,"scheduling_latency_s":0.453045} gitlab # [ 427.871005] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.342Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b39ddae2ce675522eacb8fad","created_at":"2026-10-04T19:11:01.863Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.865Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b39ddae2ce675522eacb8fad: done: 0.024504 sec","job_status":"done","queue_duration_s":0.453027,"scheduling_latency_s":0.453045,"redis_calls":7,"redis_duration_s":0.000915,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00066,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000255,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017372,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024504,"completed_at":"2026-10-04T19:11:02.342Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003637,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.914932] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.343Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1a64eaf50b2c982cb18cfd2f","created_at":"2026-10-04T19:11:01.865Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.867Z","job_size_bytes":6,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1a64eaf50b2c982cb18cfd2f: start","job_status":"start","queue_duration_s":0.476179,"scheduling_latency_s":0.476198} gitlab # [ 427.931366] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.358Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1a64eaf50b2c982cb18cfd2f","created_at":"2026-10-04T19:11:01.865Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.867Z","job_size_bytes":6,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1a64eaf50b2c982cb18cfd2f: done: 0.014337 sec","job_status":"done","queue_duration_s":0.476179,"scheduling_latency_s":0.476198,"redis_calls":7,"redis_duration_s":0.001154,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000325,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014337,"completed_at":"2026-10-04T19:11:02.358Z","load_balancing_strategy":"primary","db_duration_s":0.000335,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.974834] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.359Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a04abdc88f7a31bb3a22fc38","created_at":"2026-10-04T19:11:01.868Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.869Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a04abdc88f7a31bb3a22fc38: start","job_status":"start","queue_duration_s":0.48972,"scheduling_latency_s":0.489737} gitlab # [ 427.990634] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.382Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"a04abdc88f7a31bb3a22fc38","created_at":"2026-10-04T19:11:01.868Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.869Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-a04abdc88f7a31bb3a22fc38: done: 0.023136 sec","job_status":"done","queue_duration_s":0.48972,"scheduling_latency_s":0.489737,"redis_calls":7,"redis_duration_s":0.001193,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000332,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016464,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023136,"completed_at":"2026-10-04T19:11:02.382Z","load_balancing_strategy":"primary","db_duration_s":0.002804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.033954] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.384Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"def827140917d84d0b4c14ba","created_at":"2026-10-04T19:11:01.937Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.939Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-def827140917d84d0b4c14ba: start","job_status":"start","queue_duration_s":0.444461,"scheduling_latency_s":0.444478} gitlab # [ 428.050162] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.409Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"def827140917d84d0b4c14ba","created_at":"2026-10-04T19:11:01.937Z","trace_propagation_headers":{"sentry-trace":"2710189ac2194266ba4d18201555cbd0-adde76e7d8d94e86","baggage":"sentry-trace_id=2710189ac2194266ba4d18201555cbd0,sentry-release=Unknown"},"correlation_id":"01M44577HTB5CN5PG1WZ1J69JZ","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:01.939Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-def827140917d84d0b4c14ba: done: 0.025782 sec","job_status":"done","queue_duration_s":0.444461,"scheduling_latency_s":0.444478,"redis_calls":8,"redis_duration_s":0.001377,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001071,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000306,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025782,"completed_at":"2026-10-04T19:11:02.409Z","load_balancing_strategy":"primary","db_duration_s":0.003989,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.094319] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.410Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"219169af7284ed3b66f7d85b","created_at":"2026-10-04T19:11:02.103Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.109Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-219169af7284ed3b66f7d85b: start","job_status":"start","queue_duration_s":0.301844,"scheduling_latency_s":0.30186} gitlab # [ 428.111548] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.439Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"219169af7284ed3b66f7d85b","created_at":"2026-10-04T19:11:02.103Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.109Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-219169af7284ed3b66f7d85b: done: 0.02901 sec","job_status":"done","queue_duration_s":0.301844,"scheduling_latency_s":0.30186,"redis_calls":7,"redis_duration_s":0.001201,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02901,"completed_at":"2026-10-04T19:11:02.439Z","load_balancing_strategy":"primary","db_duration_s":0.00514,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.156034] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.441Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"65577551c868f5afb24b66e8","created_at":"2026-10-04T19:11:02.110Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.114Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-65577551c868f5afb24b66e8: start","job_status":"start","queue_duration_s":0.326499,"scheduling_latency_s":0.326517} gitlab # [ 428.173888] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.495Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"65577551c868f5afb24b66e8","created_at":"2026-10-04T19:11:02.110Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.114Z","job_size_bytes":65,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-65577551c868f5afb24b66e8: done: 0.054848 sec","job_status":"done","queue_duration_s":0.326499,"scheduling_latency_s":0.326517,"redis_calls":10,"redis_duration_s":0.0016649999999999998,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000757,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054848,"completed_at":"2026-10-04T19:11:02.495Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.219336] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.497Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7c89eb5aa824b1c35a78e087","created_at":"2026-10-04T19:11:02.185Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.190Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7c89eb5aa824b1c35a78e087: start","job_status":"start","queue_duration_s":0.307526,"scheduling_latency_s":0.307551} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 428.235801] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.663Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"7c89eb5aa824b1c35a78e087","created_at":"2026-10-04T19:11:02.185Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-1346da2bf2b14101","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.190Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-7c89eb5aa824b1c35a78e087: done: 0.165773 sec","job_status":"done","queue_duration_s":0.307526,"scheduling_latency_s":0.307551,"gitaly_calls":2,"gitaly_duration_s":0.00725,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007112,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.00118,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001107,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000308,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001598,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000306,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000448,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.130958,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12559128,"duration_s":0.165773,"completed_at":"2026-10-04T19:11:02.663Z","load_balancing_strategy":"primary","db_duration_s":0.029763,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.288793] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.665Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7d1286a8d20218c7227b127b","created_at":"2026-10-04T19:11:02.575Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-3c2e95533ba045ad","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.578Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d1286a8d20218c7227b127b: start","job_status":"start","queue_duration_s":0.08682,"scheduling_latency_s":0.086846} gitlab # [ 428.305311] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:02.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"7d1286a8d20218c7227b127b","created_at":"2026-10-04T19:11:02.575Z","trace_propagation_headers":{"sentry-trace":"744b91d8a479483e95b1d66cb50a14b9-3c2e95533ba045ad","baggage":"sentry-trace_id=744b91d8a479483e95b1d66cb50a14b9,sentry-release=Unknown"},"correlation_id":"01M44579694NVANNPWYRQ918KC","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:02.578Z","job_size_bytes":3,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-7d1286a8d20218c7227b127b: done: 0.042027 sec","job_status":"done","queue_duration_s":0.08682,"scheduling_latency_s":0.086846,"redis_calls":7,"redis_duration_s":0.0014240000000000001,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00103,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031542,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.042027,"completed_at":"2026-10-04T19:11:02.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.373780] gitaly[923]: time="2026-10-04T19:11:03.295Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457AHY8VX9RKQ0S30X3J3R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:33.293" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:03.294" grpc.time_ms=0.832 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.15 seconds) gitlab # [ 428.407094] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:03 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 97 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 429.505309] gitaly[923]: time="2026-10-04T19:11:04.426Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457BNBXDDWXNHNN3YD9RST grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:34.424" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:04.425" grpc.time_ms=0.96 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.13 seconds) gitlab # [ 429.537453] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:04 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 94 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 430.635071] gitaly[923]: time="2026-10-04T19:11:05.556Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457CRME4Q33D1DB5C0WH0X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:35.554" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:05.555" grpc.time_ms=0.922 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.13 seconds) gitlab # [ 430.668926] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:05 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 97 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 431.780426] nix-daemon[3777]: accepted connection from pid 4869, user nixbot gitlab # [ 431.784297] gitaly[923]: time="2026-10-04T19:11:06.705Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457DW0XSSGD5PAZMYNYWVF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:36.704" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:06.704" grpc.time_ms=0.796 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.801005] nix-daemon[3777]: accepted connection from pid 4835, user nixbot gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.16 seconds) gitlab # [ 431.828396] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 125 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 432.934099] gitaly[923]: time="2026-10-04T19:11:07.855Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457F072TBMV5WXDN17G1W4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:37.853" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:07.854" grpc.time_ms=0.859 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.14 seconds) gitlab # [ 432.970019] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 107 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 434.071639] gitaly[923]: time="2026-10-04T19:11:08.992Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457G3YQN3DB282TWF1P508 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:38.691" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:08.991" grpc.time_ms=0.924 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.13 seconds) gitlab # [ 434.104221] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:09 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 98 gitlab # [ 434.501991] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"44eeff641f5f98eb1fb4686a","created_at":"2026-10-04T19:11:09.371Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c01e1498fbcd078e4d1cc8620e2ea8c9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.420Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-44eeff641f5f98eb1fb4686a: start","job_status":"start","queue_duration_s":0.002291,"scheduling_latency_s":0.002329} gitlab # [ 434.610347] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.531Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"44eeff641f5f98eb1fb4686a","created_at":"2026-10-04T19:11:09.371Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c01e1498fbcd078e4d1cc8620e2ea8c9","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.420Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-44eeff641f5f98eb1fb4686a: done: 0.10886 sec","job_status":"done","queue_duration_s":0.002291,"scheduling_latency_s":0.002329,"redis_calls":11,"redis_duration_s":0.014583,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002572,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007065,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004946,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00233056,"exclusive_lock_hold_duration_s":0.0107689,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.10886,"completed_at":"2026-10-04T19:11:09.531Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028427,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.678665] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.540Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0cf3b93a0892335e12af2aa1","created_at":"2026-10-04T19:11:09.428Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9dbba8f28ba454e826beec3dfa5d0cf4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:11:09.436Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0cf3b93a0892335e12af2aa1: start","job_status":"start","queue_duration_s":0.103623,"scheduling_latency_s":0.103655} gitlab # [ 434.700301] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.606Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 434.706305] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"0cf3b93a0892335e12af2aa1","created_at":"2026-10-04T19:11:09.428Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9dbba8f28ba454e826beec3dfa5d0cf4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-04T19:11:09.436Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-0cf3b93a0892335e12af2aa1: done: 0.08716 sec","job_status":"done","queue_duration_s":0.103623,"scheduling_latency_s":0.103655,"redis_calls":9,"redis_duration_s":0.030773,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.024838,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003557,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002378,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.08716,"completed_at":"2026-10-04T19:11:09.627Z","load_balancing_strategy":"primary","db_duration_s":0.002559,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.760795] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.635Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"a6c8024b1d0e32195f417cca","created_at":"2026-10-04T19:11:09.449Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.462Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-a6c8024b1d0e32195f417cca: start","job_status":"start","queue_duration_s":0.172516,"scheduling_latency_s":0.172551} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 435.056503] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleOldPipelinesRemovalCronWorker","jid":"a6c8024b1d0e32195f417cca","created_at":"2026-10-04T19:11:09.449Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5cd05815835513d57d02434c59416c33dd1bebcfa3138e1542ab850c977747b0","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.462Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleOldPipelinesRemovalCronWorker JID-a6c8024b1d0e32195f417cca: done: 0.342394 sec","job_status":"done","queue_duration_s":0.172516,"scheduling_latency_s":0.172551,"redis_calls":28,"redis_duration_s":0.20739,"redis_read_bytes":36,"redis_write_bytes":10583,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003702,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":680,"redis_queues_calls":2,"redis_queues_duration_s":0.002025,"redis_queues_read_bytes":3,"redis_queues_write_bytes":8295,"redis_queues_metadata_calls":16,"redis_queues_metadata_duration_s":0.192239,"redis_queues_metadata_read_bytes":24,"redis_queues_metadata_write_bytes":1162,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.009424,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":446,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053162,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.16977284,"duration_s":0.342394,"completed_at":"2026-10-04T19:11:09.977Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004054,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.125564] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:09.982Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c983baf1942443df4666fbe4","created_at":"2026-10-04T19:11:09.470Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"850a3d45274f6f23de59a52820604213","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:11:09.477Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c983baf1942443df4666fbe4: start","job_status":"start","queue_duration_s":0.504959,"scheduling_latency_s":0.50499} gitlab # [ 435.180193] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.101Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c983baf1942443df4666fbe4","created_at":"2026-10-04T19:11:09.470Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"850a3d45274f6f23de59a52820604213","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-04T19:11:09.477Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c983baf1942443df4666fbe4: done: 0.119159 sec","job_status":"done","queue_duration_s":0.504959,"scheduling_latency_s":0.50499,"redis_calls":9,"redis_duration_s":0.028318,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.02044,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001983,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.037578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119159,"completed_at":"2026-10-04T19:11:10.101Z","load_balancing_strategy":"primary","db_duration_s":0.01105,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.237827] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"858ae5d5586f24728eebb17e","created_at":"2026-10-04T19:11:09.489Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb0e5a471c6cb37fdede97333c949560","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.499Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-858ae5d5586f24728eebb17e: start","job_status":"start","queue_duration_s":0.61079,"scheduling_latency_s":0.610828} gitlab # [ 435.254084] gitaly[923]: time="2026-10-04T19:11:10.149Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457H7TBCQ9A5C8G2PHBR82 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:40.145" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:10.146" grpc.time_ms=3.364 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.16 seconds) gitlab # [ 435.268608] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:10 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 112 gitlab # [ 435.325523] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.246Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"858ae5d5586f24728eebb17e","created_at":"2026-10-04T19:11:09.489Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bb0e5a471c6cb37fdede97333c949560","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.499Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-858ae5d5586f24728eebb17e: done: 0.136215 sec","job_status":"done","queue_duration_s":0.61079,"scheduling_latency_s":0.610828,"redis_calls":13,"redis_duration_s":0.005586,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001943,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001794,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.035,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.023,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.069919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00104684,"exclusive_lock_hold_duration_s":0.0776827,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.136215,"completed_at":"2026-10-04T19:11:10.246Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.040161,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.384216] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"825911c5be4f48499051e09f","created_at":"2026-10-04T19:11:09.512Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed767360bdb8da941ec260b69339a03e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.528Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-825911c5be4f48499051e09f: start","job_status":"start","queue_duration_s":0.720716,"scheduling_latency_s":0.720741} gitlab # [ 435.572757] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.493Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"825911c5be4f48499051e09f","created_at":"2026-10-04T19:11:09.512Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ed767360bdb8da941ec260b69339a03e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.528Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-825911c5be4f48499051e09f: done: 0.244975 sec","job_status":"done","queue_duration_s":0.720716,"scheduling_latency_s":0.720741,"redis_calls":11,"redis_duration_s":0.002355,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000399,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000876,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.031,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.202948,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00060826,"exclusive_lock_hold_duration_s":0.21998952,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.244975,"completed_at":"2026-10-04T19:11:10.493Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047027,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.641564] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.495Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5aea59e25366478359c9eb1a","created_at":"2026-10-04T19:11:09.537Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.545Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5aea59e25366478359c9eb1a: start","job_status":"start","queue_duration_s":0.95004,"scheduling_latency_s":0.950057} gitlab # [ 435.660994] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.532Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"5aea59e25366478359c9eb1a","created_at":"2026-10-04T19:11:09.537Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.545Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-5aea59e25366478359c9eb1a: done: 0.037304 sec","job_status":"done","queue_duration_s":0.95004,"scheduling_latency_s":0.950057,"redis_calls":18,"redis_duration_s":0.003056,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000767,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000231,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001525,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000533,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026642,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.037304,"completed_at":"2026-10-04T19:11:10.532Z","load_balancing_strategy":"primary","db_duration_s":0.002733,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.714901] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.533Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7a85f5d4b9ffd7decfcf870d","created_at":"2026-10-04T19:11:09.556Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ab52ed1a77a07d23e43d0eab0aff148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.564Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7a85f5d4b9ffd7decfcf870d: start","job_status":"start","queue_duration_s":0.969535,"scheduling_latency_s":0.969551} gitlab # [ 435.730096] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.545Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 435.733415] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.554Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"7a85f5d4b9ffd7decfcf870d","created_at":"2026-10-04T19:11:09.556Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0ab52ed1a77a07d23e43d0eab0aff148","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.564Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-7a85f5d4b9ffd7decfcf870d: done: 0.020958 sec","job_status":"done","queue_duration_s":0.969535,"scheduling_latency_s":0.969551,"redis_calls":9,"redis_duration_s":0.0017230000000000001,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000435,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020958,"completed_at":"2026-10-04T19:11:10.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.777158] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.555Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2f48ad02b2cdb926f0abab42","created_at":"2026-10-04T19:11:09.600Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fb755f6b662efc3341a6c49b51d9bd6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.613Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2f48ad02b2cdb926f0abab42: start","job_status":"start","queue_duration_s":0.942776,"scheduling_latency_s":0.942791} gitlab # [ 435.791766] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"2f48ad02b2cdb926f0abab42","created_at":"2026-10-04T19:11:09.600Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fb755f6b662efc3341a6c49b51d9bd6","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.613Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-2f48ad02b2cdb926f0abab42: done: 0.024867 sec","job_status":"done","queue_duration_s":0.942776,"scheduling_latency_s":0.942791,"redis_calls":9,"redis_duration_s":0.001528,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024867,"completed_at":"2026-10-04T19:11:10.580Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.835749] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"deb9c8979dda007091a489a1","created_at":"2026-10-04T19:11:09.623Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a6c034c85d1c8334ec3527246d7796","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.634Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-deb9c8979dda007091a489a1: start","job_status":"start","queue_duration_s":0.947218,"scheduling_latency_s":0.947234} gitlab # [ 435.859665] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.614Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"deb9c8979dda007091a489a1","created_at":"2026-10-04T19:11:09.623Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"52a6c034c85d1c8334ec3527246d7796","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.634Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-deb9c8979dda007091a489a1: done: 0.032491 sec","job_status":"done","queue_duration_s":0.947218,"scheduling_latency_s":0.947234,"redis_calls":12,"redis_duration_s":0.0022340000000000003,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000373,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000818,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021727,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00982276,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.032491,"completed_at":"2026-10-04T19:11:10.614Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00355,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 435.934565] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.615Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c264392f1f3b723679301331","created_at":"2026-10-04T19:11:09.664Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"830367fbf6addfe05e849a6365bcc28d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.671Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c264392f1f3b723679301331: start","job_status":"start","queue_duration_s":0.944072,"scheduling_latency_s":0.944087} gitlab # [ 435.954390] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.645Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"c264392f1f3b723679301331","created_at":"2026-10-04T19:11:09.664Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"830367fbf6addfe05e849a6365bcc28d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.671Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-c264392f1f3b723679301331: done: 0.029779 sec","job_status":"done","queue_duration_s":0.944072,"scheduling_latency_s":0.944087,"redis_calls":9,"redis_duration_s":0.001627,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000341,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000904,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029779,"completed_at":"2026-10-04T19:11:10.645Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009378,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.004887] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.646Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"22b99ae8bb70de681c6f8a32","created_at":"2026-10-04T19:11:09.675Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d3f33db407c8ba58e942188723e2456","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.688Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-22b99ae8bb70de681c6f8a32: start","job_status":"start","queue_duration_s":0.958483,"scheduling_latency_s":0.958499} gitlab # [ 436.018913] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.760Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"22b99ae8bb70de681c6f8a32","created_at":"2026-10-04T19:11:09.675Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1d3f33db407c8ba58e942188723e2456","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.688Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-22b99ae8bb70de681c6f8a32: done: 0.113867 sec","job_status":"done","queue_duration_s":0.958483,"scheduling_latency_s":0.958499,"redis_calls":70,"redis_duration_s":0.012492,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001694,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000717,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009609,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.113867,"completed_at":"2026-10-04T19:11:10.760Z","load_balancing_strategy":"primary","db_duration_s":0.001607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.063996] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.761Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ab791b62e8ba88976e089c92","created_at":"2026-10-04T19:11:09.702Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0deb8e287194efa08b7fab7f236ff8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.716Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ab791b62e8ba88976e089c92: start","job_status":"start","queue_duration_s":1.045288,"scheduling_latency_s":1.045304} gitlab # [ 436.079725] nix-daemon[3777]: accepted connection from pid 4835, user nixbot gitlab # [ 436.080812] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.782Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"ab791b62e8ba88976e089c92","created_at":"2026-10-04T19:11:09.702Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b0deb8e287194efa08b7fab7f236ff8d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.716Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-ab791b62e8ba88976e089c92: done: 0.020897 sec","job_status":"done","queue_duration_s":1.045288,"scheduling_latency_s":1.045304,"redis_calls":11,"redis_duration_s":0.001793,"redis_read_bytes":9,"redis_write_bytes":1386,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":610,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000333,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013016,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020897,"completed_at":"2026-10-04T19:11:10.782Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000786,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.124315] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"194e3496d0f2bbde09e7f3ca","created_at":"2026-10-04T19:11:09.748Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10ec1e8301fdc68f8d634ca34ee97bb0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.759Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-194e3496d0f2bbde09e7f3ca: start","job_status":"start","queue_duration_s":1.024303,"scheduling_latency_s":1.024318} gitlab # [ 436.139458] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.801Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"194e3496d0f2bbde09e7f3ca","created_at":"2026-10-04T19:11:09.748Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"10ec1e8301fdc68f8d634ca34ee97bb0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.759Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-194e3496d0f2bbde09e7f3ca: done: 0.017488 sec","job_status":"done","queue_duration_s":1.024303,"scheduling_latency_s":1.024318,"redis_calls":9,"redis_duration_s":0.001459,"redis_read_bytes":7,"redis_write_bytes":1192,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":361,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000837,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010842,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017488,"completed_at":"2026-10-04T19:11:10.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.183450] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"18324d70b1a316eb11a8c923","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-18324d70b1a316eb11a8c923: start","job_status":"start","queue_duration_s":1.016445,"scheduling_latency_s":1.016461} gitlab # [ 436.197461] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.837Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"18324d70b1a316eb11a8c923","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-18324d70b1a316eb11a8c923: done: 0.035303 sec","job_status":"done","queue_duration_s":1.016445,"scheduling_latency_s":1.016461,"redis_calls":16,"redis_duration_s":0.003239,"redis_read_bytes":14,"redis_write_bytes":1526,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00039,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000963,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":423,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001886,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":802,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017964,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032736,"duration_s":0.035303,"completed_at":"2026-10-04T19:11:10.837Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007403,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.239959] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c03ea932ed06a64ba4d87be9","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c03ea932ed06a64ba4d87be9: start","job_status":"start","queue_duration_s":1.053364,"scheduling_latency_s":1.053386} gitlab # [ 436.254194] gitaly[923]: time="2026-10-04T19:11:11.098Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457J5APFV36STQ20W019CD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:41.096" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:11.097" grpc.time_ms=0.827 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 436.267443] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c03ea932ed06a64ba4d87be9","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c03ea932ed06a64ba4d87be9: done: 0.021784 sec","job_status":"done","queue_duration_s":1.053364,"scheduling_latency_s":1.053386,"redis_calls":13,"redis_duration_s":0.002529,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000829,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0017,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014445,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021784,"completed_at":"2026-10-04T19:11:10.861Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00089,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.308582] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.863Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a3de8d39ad774f8cfe95a828","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a3de8d39ad774f8cfe95a828: start","job_status":"start","queue_duration_s":1.076996,"scheduling_latency_s":1.077017} gitlab # [ 436.322390] gitaly[923]: time="2026-10-04T19:11:11.158Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4457J5APFV36STQ20W019CD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:21.147" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:11.148" grpc.time_ms=10.28 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.336754] nixbot[3615]: {"ts": "2026-10-04T19:11:11.222Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf \"HTTP/1.1 201 Created\""} gitlab # [ 436.339847] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 202 gitlab # [ 436.342715] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"a3de8d39ad774f8cfe95a828","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-a3de8d39ad774f8cfe95a828: done: 0.020962 sec","job_status":"done","queue_duration_s":1.076996,"scheduling_latency_s":1.077017,"redis_calls":13,"redis_duration_s":0.0025770000000000003,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000745,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020962,"completed_at":"2026-10-04T19:11:10.884Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00108,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.12 seconds) gitlab # [ 436.383839] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.886Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2a210e402e01c6989415df2b","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2a210e402e01c6989415df2b: start","job_status":"start","queue_duration_s":1.0998,"scheduling_latency_s":1.099819} gitlab # [ 436.397670] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.902Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"2a210e402e01c6989415df2b","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-2a210e402e01c6989415df2b: done: 0.016873 sec","job_status":"done","queue_duration_s":1.0998,"scheduling_latency_s":1.099819,"redis_calls":13,"redis_duration_s":0.001941,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001252,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016873,"completed_at":"2026-10-04T19:11:10.902Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000823,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.438730] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"193353d586f5c8d33fa9ed06","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-193353d586f5c8d33fa9ed06: start","job_status":"start","queue_duration_s":1.118278,"scheduling_latency_s":1.118296} gitlab # [ 436.452740] gitaly[923]: time="2026-10-04T19:11:11.283Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457JC1895CW8JTA4MAT8YJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:41.281" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:11.282" grpc.time_ms=0.621 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.465968] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:11 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 74 gitlab # [ 436.468714] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"193353d586f5c8d33fa9ed06","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-193353d586f5c8d33fa9ed06: done: 0.016866 sec","job_status":"done","queue_duration_s":1.118278,"scheduling_latency_s":1.118296,"redis_calls":13,"redis_duration_s":0.0019450000000000001,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001268,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010309,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016866,"completed_at":"2026-10-04T19:11:10.921Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001065,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.510677] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"414d78c8d25a583c8696ab13","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-414d78c8d25a583c8696ab13: start","job_status":"start","queue_duration_s":1.136388,"scheduling_latency_s":1.136409} gitlab # [ 436.524508] gitaly[923]: time="2026-10-04T19:11:11.315Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457JBVTPVXDGBT25ME39HA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:41.314" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:11.314" grpc.time_ms=0.531 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.537550] nixbot[3615]: {"ts": "2026-10-04T19:11:11.419Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf \"HTTP/1.1 201 Created\""} gitlab # [ 436.540395] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"414d78c8d25a583c8696ab13","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-414d78c8d25a583c8696ab13: done: 0.016214 sec","job_status":"done","queue_duration_s":1.136388,"scheduling_latency_s":1.136409,"redis_calls":13,"redis_duration_s":0.002467,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001744,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010109,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016214,"completed_at":"2026-10-04T19:11:10.938Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000994,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.581576] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9ee68f8971e26fe672be7df3","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9ee68f8971e26fe672be7df3: start","job_status":"start","queue_duration_s":1.153735,"scheduling_latency_s":1.15375} gitlab # [ 436.595275] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 190 gitlab # [ 436.597941] gitaly[923]: time="2026-10-04T19:11:11.362Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4457JBVTPVXDGBT25ME39HA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:21.234" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:11.355" grpc.time_ms=7.366 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.611857] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"9ee68f8971e26fe672be7df3","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-9ee68f8971e26fe672be7df3: done: 0.015311 sec","job_status":"done","queue_duration_s":1.153735,"scheduling_latency_s":1.15375,"redis_calls":13,"redis_duration_s":0.002055,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001417,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015311,"completed_at":"2026-10-04T19:11:10.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.653137] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"80a310fdf14a54cf98c0b221","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-80a310fdf14a54cf98c0b221: start","job_status":"start","queue_duration_s":1.170217,"scheduling_latency_s":1.170233} gitlab # [ 436.666833] gitaly[923]: time="2026-10-04T19:11:11.485Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457JJ4CWXCZZA803M4A9HP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-04T19:11:41.484" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-04T19:11:11.484" grpc.time_ms=0.601 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.679976] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.972Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"80a310fdf14a54cf98c0b221","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-80a310fdf14a54cf98c0b221: done: 0.0163 sec","job_status":"done","queue_duration_s":1.170217,"scheduling_latency_s":1.170233,"redis_calls":13,"redis_duration_s":0.002092,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001445,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0163,"completed_at":"2026-10-04T19:11:10.972Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.723736] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.973Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8b71e5062e5e97138f296feb","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8b71e5062e5e97138f296feb: start","job_status":"start","queue_duration_s":1.187689,"scheduling_latency_s":1.187703} gitlab # [ 436.737887] gitaly[923]: time="2026-10-04T19:11:11.531Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4457JJ4CWXCZZA803M4A9HP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-04T19:11:21.063" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-04T19:11:11.524" grpc.time_ms=7.543 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.751936] nixbot[3615]: {"ts": "2026-10-04T19:11:11.594Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf \"HTTP/1.1 201 Created\""} gitlab # [ 436.754983] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:11 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/459fc935a5411646cc3e2e58cd0153ee2c26ddcf HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 164 gitlab # [ 436.757674] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.989Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"8b71e5062e5e97138f296feb","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-8b71e5062e5e97138f296feb: done: 0.015239 sec","job_status":"done","queue_duration_s":1.187689,"scheduling_latency_s":1.187703,"redis_calls":13,"redis_duration_s":0.0018909999999999999,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015239,"completed_at":"2026-10-04T19:11:10.989Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.798576] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:10.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c2ab4489e56f662652950b83","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c2ab4489e56f662652950b83: start","job_status":"start","queue_duration_s":1.204073,"scheduling_latency_s":1.204089} gitlab # [ 436.812418] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.006Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::DestroyOldPipelinesWorker","args":[],"jid":"c2ab4489e56f662652950b83","created_at":"2026-10-04T19:11:09.711Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-5667ed56a486474c","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleOldPipelinesRemovalCronWorker","correlation_id":"f67bd9e2a39be8726ed61d2e00294e13","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_integration","meta.client_id":"ip/","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.786Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::DestroyOldPipelinesWorker JID-c2ab4489e56f662652950b83: done: 0.016112 sec","job_status":"done","queue_duration_s":1.204073,"scheduling_latency_s":1.204089,"redis_calls":13,"redis_duration_s":0.002463,"redis_read_bytes":10,"redis_write_bytes":1149,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000814,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":347,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001649,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":802,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016112,"completed_at":"2026-10-04T19:11:11.006Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.853302] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.008Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0d256b9b238a18a5494d4c23","created_at":"2026-10-04T19:11:09.776Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37d15fd9f42793e600a37e2453151664","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.788Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0d256b9b238a18a5494d4c23: start","job_status":"start","queue_duration_s":1.219933,"scheduling_latency_s":1.21995} gitlab # [ 436.869246] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"0d256b9b238a18a5494d4c23","created_at":"2026-10-04T19:11:09.776Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"37d15fd9f42793e600a37e2453151664","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.788Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-0d256b9b238a18a5494d4c23: done: 0.043786 sec","job_status":"done","queue_duration_s":1.219933,"scheduling_latency_s":1.21995,"redis_calls":11,"redis_duration_s":0.003309,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000908,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001026,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02052,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00070984,"exclusive_lock_hold_duration_s":0.01639528,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.043786,"completed_at":"2026-10-04T19:11:11.051Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015965,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.916572] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"65225e26d3b39be11c776f5a","created_at":"2026-10-04T19:11:09.964Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"871eb5981b656bbe382fa4bbe8a98d82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.980Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-65225e26d3b39be11c776f5a: start","job_status":"start","queue_duration_s":1.072929,"scheduling_latency_s":1.072946} gitlab # [ 436.930916] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.080Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"65225e26d3b39be11c776f5a","created_at":"2026-10-04T19:11:09.964Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"871eb5981b656bbe382fa4bbe8a98d82","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.980Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-65225e26d3b39be11c776f5a: done: 0.026682 sec","job_status":"done","queue_duration_s":1.072929,"scheduling_latency_s":1.072946,"redis_calls":11,"redis_duration_s":0.0021349999999999997,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.0005,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000797,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016538,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052842,"exclusive_lock_hold_duration_s":0.00735016,"duration_s":0.026682,"completed_at":"2026-10-04T19:11:11.080Z","load_balancing_strategy":"primary","db_duration_s":0.003609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.974570] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.081Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"088df21d98113cd7cbf627f9","created_at":"2026-10-04T19:11:09.986Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db3dcb1bf1552cc0e670d084c15c4280","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.993Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-088df21d98113cd7cbf627f9: start","job_status":"start","queue_duration_s":1.088359,"scheduling_latency_s":1.088375} gitlab # [ 436.989140] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"088df21d98113cd7cbf627f9","created_at":"2026-10-04T19:11:09.986Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"db3dcb1bf1552cc0e670d084c15c4280","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:09.993Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-088df21d98113cd7cbf627f9: done: 0.033479 sec","job_status":"done","queue_duration_s":1.088359,"scheduling_latency_s":1.088375,"redis_calls":9,"redis_duration_s":0.001632,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000379,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.033479,"completed_at":"2026-10-04T19:11:11.115Z","load_balancing_strategy":"primary","db_duration_s":0.011204,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.033134] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.116Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"491330d12559101bad74e062","created_at":"2026-10-04T19:11:09.998Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55e9e66be92b525b6a09c3278fa70fdd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.011Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-491330d12559101bad74e062: start","job_status":"start","queue_duration_s":1.104926,"scheduling_latency_s":1.104943} gitlab # [ 437.047734] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"491330d12559101bad74e062","created_at":"2026-10-04T19:11:09.998Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"55e9e66be92b525b6a09c3278fa70fdd","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.011Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-491330d12559101bad74e062: done: 0.336655 sec","job_status":"done","queue_duration_s":1.104926,"scheduling_latency_s":1.104943,"redis_calls":564,"redis_duration_s":0.066682,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000318,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.065571,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.30413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.336655,"completed_at":"2026-10-04T19:11:11.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000457,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.091335] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ef147e4602ec450d2ec58e99","created_at":"2026-10-04T19:11:10.051Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13928ca1d56d89824a775615a6ad4eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.074Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ef147e4602ec450d2ec58e99: start","job_status":"start","queue_duration_s":1.379878,"scheduling_latency_s":1.379902} gitlab # [ 437.106605] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.469Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 437.111104] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:11 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 437.113356] gitaly[923]: time="2026-10-04T19:11:11.951Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M4457K09378P54PPBFJW1PQP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:11.943" grpc.time_ms=7.908 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 437.127045] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"ef147e4602ec450d2ec58e99","created_at":"2026-10-04T19:11:10.051Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"13928ca1d56d89824a775615a6ad4eff","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.074Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-ef147e4602ec450d2ec58e99: done: 0.024377 sec","job_status":"done","queue_duration_s":1.379878,"scheduling_latency_s":1.379902,"redis_calls":9,"redis_duration_s":0.001346,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000709,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013868,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024377,"completed_at":"2026-10-04T19:11:11.479Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000551,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.171326] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a0203bcd0986ad12ae267a2f","created_at":"2026-10-04T19:11:10.096Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a41a060f8003d75e2f12f57964671047","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.112Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a0203bcd0986ad12ae267a2f: start","job_status":"start","queue_duration_s":1.367837,"scheduling_latency_s":1.367854} gitlab # [ 437.186769] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.493Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 437.190818] gitlab-workhorse[822]: time="2026-10-04T19:11:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 437.192769] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 85 gitlab # [ 437.194699] gitaly[923]: time="2026-10-04T19:11:12.037Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4457K2HW98TX079QXFVF18S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.027" pid=923 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 437.205483] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.503Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a0203bcd0986ad12ae267a2f","created_at":"2026-10-04T19:11:10.096Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a41a060f8003d75e2f12f57964671047","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.112Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a0203bcd0986ad12ae267a2f: done: 0.023478 sec","job_status":"done","queue_duration_s":1.367837,"scheduling_latency_s":1.367854,"redis_calls":9,"redis_duration_s":0.001282,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000287,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000685,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00031,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013299,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023478,"completed_at":"2026-10-04T19:11:11.503Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000587,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.249581] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.505Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"08354837ae80eee04878e199","created_at":"2026-10-04T19:11:10.142Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f1a0e1da971bceb20744c094221d587","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.171Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-08354837ae80eee04878e199: start","job_status":"start","queue_duration_s":1.333527,"scheduling_latency_s":1.333544} gitlab # [ 437.264945] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.518Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 437.269214] gitaly[923]: time="2026-10-04T19:11:12.037Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457K2HW98TX079QXFVF18S grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.027" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 437.283202] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:12 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 75 gitlab # [ 437.285429] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"08354837ae80eee04878e199","created_at":"2026-10-04T19:11:10.142Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0f1a0e1da971bceb20744c094221d587","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.171Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-08354837ae80eee04878e199: done: 0.021843 sec","job_status":"done","queue_duration_s":1.333527,"scheduling_latency_s":1.333544,"redis_calls":9,"redis_duration_s":0.0017100000000000001,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012354,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021843,"completed_at":"2026-10-04T19:11:11.526Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0008,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.330173] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"866d6cb32751fb81063f7312","created_at":"2026-10-04T19:11:10.196Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e5906e878cc74369ddf5ecb22ccfb3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:11:10.209Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-866d6cb32751fb81063f7312: start","job_status":"start","queue_duration_s":1.319338,"scheduling_latency_s":1.319357} gitlab # [ 437.344075] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.544Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 437.347443] gitaly[923]: time="2026-10-04T19:11:12.038Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M4457K2HW98TX079QXFVF18S grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.027" grpc.time_ms=10.84 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 437.362931] gitlab-workhorse[822]: time="2026-10-04T19:11:12Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 437.364905] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/04:19:11:12 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 74 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses gitlab # [ 437.366855] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"866d6cb32751fb81063f7312","created_at":"2026-10-04T19:11:10.196Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-8aeed25bf12244cc","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e5e5906e878cc74369ddf5ecb22ccfb3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-04T19:11:10.209Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-866d6cb32751fb81063f7312: done: 0.023696 sec","job_status":"done","queue_duration_s":1.319338,"scheduling_latency_s":1.319357,"redis_calls":9,"redis_duration_s":0.001794,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000303,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001157,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000334,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023696,"completed_at":"2026-10-04T19:11:11.552Z","load_balancing_strategy":"primary","db_duration_s":0.003437,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.408336] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.555Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"375eda750eb3d118e72453ef","created_at":"2026-10-04T19:11:10.511Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-49b6f5deb37344c6","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.524Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-375eda750eb3d118e72453ef: start","job_status":"start","queue_duration_s":1.030919,"scheduling_latency_s":1.030937} gitlab # [ 437.421236] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.571Z","message":"No existing merge request to be cleaned up."} gitlab # [ 437.423010] gitaly[923]: time="2026-10-04T19:11:12.148Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M4457K69CX5G8GP37JJVDPPN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.139" grpc.time_ms=9.171 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 437.436831] gitaly[923]: time="2026-10-04T19:11:12.224Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M4457K8PCB2RSGZ0B29XV7NZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.214" pid=923 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 437.447335] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.586Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"375eda750eb3d118e72453ef","created_at":"2026-10-04T19:11:10.511Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-49b6f5deb37344c6","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.524Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-375eda750eb3d118e72453ef: done: 0.031506 sec","job_status":"done","queue_duration_s":1.030919,"scheduling_latency_s":1.030937,"redis_calls":12,"redis_duration_s":0.001823,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000608,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019568,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031506,"completed_at":"2026-10-04T19:11:11.586Z","load_balancing_strategy":"primary","db_duration_s":0.005247,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.489510] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.589Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf2c7b08a685a4d492cf23e0","created_at":"2026-10-04T19:11:10.511Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-49b6f5deb37344c6","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.524Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf2c7b08a685a4d492cf23e0: start","job_status":"start","queue_duration_s":1.064445,"scheduling_latency_s":1.064462} gitlab # [ 437.502294] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.600Z","message":"No existing merge request to be cleaned up."} gitlab # [ 437.504085] gitaly[923]: time="2026-10-04T19:11:12.224Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M4457K8PCB2RSGZ0B29XV7NZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-04T19:11:12.214" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=923 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/459fc935a5411646cc3e2e58cd0153ee2c26ddcf/statuses, in 0.16 seconds) (finished: subtest: eval and build post commit statuses, in 10.46 seconds) (finished: run the VM test script, in 505.96 seconds) gitlab # [ 437.517594] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.609Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"bf2c7b08a685a4d492cf23e0","created_at":"2026-10-04T19:11:10.511Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-49b6f5deb37344c6","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.524Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-bf2c7b08a685a4d492cf23e0: done: 0.020457 sec","job_status":"done","queue_duration_s":1.064445,"scheduling_latency_s":1.064462,"redis_calls":10,"redis_duration_s":0.001519,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020457,"completed_at":"2026-10-04T19:11:11.609Z","load_balancing_strategy":"primary","db_duration_s":0.002031,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} test script finished in 505.98s cleanup kill QemuMachine (pid 46) gitlab # [ 437.558321] sidekiq-cluster[3494]: {"severity":"INFO","time":"2026-10-04T19:11:11.610Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ce620a6cb445657bf0f39214","created_at":"2026-10-04T19:11:10.511Z","trace_propagation_headers":{"sentry-trace":"b3d55a37577c4411a6f831d0a13f756e-49b6f5deb37344c6","baggage":"sentry-trace_id=b3d55a37577c4411a6f831d0a13f756e,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"d0bdbf5325ba48c4d4386b08fa8892d6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-04T19:11:10.524Z","job_size_bytes":2,"pid":3494,"sidekiq_tid":"m8r2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ce620a6cb445657bf0f39214: start","job_status":"start","queue_duration_s":1.086388,"scheduling_latency_s":1.086405} gitlab # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.59 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-04T19:09:33.615Z', 'push_events': True, 'tag_push_events': True, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-04T19:11:01.825618Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}] [] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': '2026-10-04T19:11:11.188Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 8s (1 warning)', 'created_at': '2026-10-04T19:11:02.175Z', 'started_at': None, 'finished_at': '2026-10-04T19:11:11.188Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '459fc935a5411646cc3e2e58cd0153ee2c26ddcf', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-04T19:11:11.392Z', 'started_at': None, 'finished_at': '2026-10-04T19:11:11.559Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]