tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.CMlnZgLaUC', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 71.63 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: 04bc9a25-9c66-4e7c-80e3-817de474bdbb gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/apgr4bbsy6asy7kwdy25hr9g9xymxn1n-nixos-system-gitlab-test/init regInfo=/nix/store/5pg8qbb8jblm9ycglvpib0nk6x937avp-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/5pg8qbb8jblm9ycglvpib0nk6x937avp-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000029] arm-pv: using stolen time PV gitlab # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000545] Console: colour dummy device 80x25 gitlab # [ 0.000552] printk: legacy console [tty0] enabled gitlab # [ 0.000736] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000743] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000828] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000953] landlock: Up and running. gitlab # [ 0.000956] Yama: becoming mindful. gitlab # [ 0.001407] LSM support for eBPF active gitlab # [ 0.001589] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001705] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.003111] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003791] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003795] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003943] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004970] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.005064] EFI services will not be available. gitlab # [ 0.005264] smp: Bringing up secondary CPUs ... gitlab # [ 0.005936] Detected PIPT I-cache on CPU1 gitlab # [ 0.006038] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.006171] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.006301] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.007432] Detected PIPT I-cache on CPU2 gitlab # [ 0.007542] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007675] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.007804] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008867] Detected PIPT I-cache on CPU3 gitlab # [ 0.008984] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.009116] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.009259] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009772] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009798] SMP: Total of 4 processors activated. gitlab # [ 0.009801] CPU: All CPU(s) started at EL1 gitlab # [ 0.009810] CPU features: detected: Branch Target Identification gitlab # [ 0.009814] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009817] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009821] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009825] CPU features: detected: Common not Private translations gitlab # [ 0.009827] CPU features: detected: CRC32 instructions gitlab # [ 0.009830] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009833] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009836] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009839] CPU features: detected: E0PD gitlab # [ 0.009842] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009845] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009848] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009851] CPU features: detected: Fine Grained Traps gitlab # [ 0.009854] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009858] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009861] CPU features: detected: LSE atomic instructions gitlab # [ 0.009864] CPU features: detected: Privileged Access Never gitlab # [ 0.009866] CPU features: detected: PMUv3 gitlab # [ 0.009869] CPU features: detected: RAS Extension Support gitlab # [ 0.009871] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009874] CPU features: detected: Random Number Generator gitlab # [ 0.009876] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009878] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009881] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009885] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.010015] alternatives: applying system-wide alternatives gitlab # [ 0.013003] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.013237] Memory: 5963824K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.014723] devtmpfs: initialized gitlab # [ 0.018847] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.018915] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.019136] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.019140] 0 pages in range for non-PLT usage gitlab # [ 0.019142] 508288 pages in range for PLT usage gitlab # [ 0.019259] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.020049] DMI not present or invalid. gitlab # [ 0.023274] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.025726] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.026008] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.028055] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.028078] audit: initializing netlink subsys (disabled) gitlab # [ 0.028445] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.028938] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.028940] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.028945] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.028948] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.028950] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.029051] cpuidle: using governor ladder gitlab # [ 0.029140] cpuidle: using governor menu gitlab # [ 0.029342] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.029555] ASID allocator initialised with 65536 entries gitlab # [ 0.030691] Serial: AMBA PL011 UART driver gitlab # [ 0.035779] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.036111] printk: console [ttyAMA0] enabled gitlab # [ 0.048015] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.048033] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.048043] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.048052] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.048060] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.048068] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.048076] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.048084] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.059670] fbcon: Taking over console gitlab # [ 0.059682] ACPI: Interpreter disabled. gitlab # [ 0.060689] iommu: Default domain type: Translated gitlab # [ 0.060707] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.070653] SCSI subsystem initialized gitlab # [ 0.071036] usbcore: registered new interface driver usbfs gitlab # [ 0.071061] usbcore: registered new interface driver hub gitlab # [ 0.071077] usbcore: registered new device driver usb gitlab # [ 0.071359] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.071362] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.071378] PTP clock support registered gitlab # [ 0.071416] EDAC MC: Ver: 3.0.0 gitlab # [ 0.071945] scmi_core: SCMI protocol bus registered gitlab # [ 0.072896] FPGA manager framework gitlab # [ 0.073703] vgaarb: loaded gitlab # [ 0.075510] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.075937] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.075960] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.076326] netfs: FS-Cache loaded gitlab # [ 0.076430] pnp: PnP ACPI: disabled gitlab # [ 0.089414] NET: Registered PF_INET protocol family gitlab # [ 0.092129] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.155022] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.155100] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.155166] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.155680] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.156628] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.156819] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.156910] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.157105] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.157391] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.157432] NET: Registered PF_XDP protocol family gitlab # [ 0.157457] PCI: CLS 0 bytes, default 64 gitlab # [ 0.157816] Trying to unpack rootfs image as initramfs... gitlab # [ 0.167721] kvm [1]: HYP mode not available gitlab # [ 0.241529] Initialise system trusted keyrings gitlab # [ 0.242051] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.242869] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.243023] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.258321] Key type asymmetric registered gitlab # [ 0.258337] Asymmetric key parser 'x509' registered gitlab # [ 0.258424] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.258646] io scheduler mq-deadline registered gitlab # [ 0.258651] io scheduler kyber registered gitlab # [ 0.265547] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.267607] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.268205] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.268225] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.268239] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.268249] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.268286] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.268315] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.268430] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.268440] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.268446] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.268451] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.268456] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.268544] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.269399] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.269808] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.269828] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.269872] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.269891] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.270810] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.271192] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.271211] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.271254] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.272122] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.272479] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.272499] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.272549] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.273382] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.273735] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.273753] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.273802] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.274617] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.274978] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.274997] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.275044] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.275854] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.276159] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.276177] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.276217] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.277047] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.277423] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.277442] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.277490] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.277509] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.278359] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.278725] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.278766] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.279597] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.279902] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.279944] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.280669] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.281029] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.281431] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.281839] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.281858] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.281899] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.282691] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.283058] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.283077] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.283131] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.284188] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.284198] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.284204] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.284285] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.284361] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.284442] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.284524] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.284604] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.284687] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.284764] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.284846] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.284935] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.285025] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.285093] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.285121] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.285148] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.285187] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.285216] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.285244] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.285271] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.285299] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.285328] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.285356] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.285385] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.285412] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.285446] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.285472] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.285498] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.285524] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.285550] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.285581] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.285608] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.285640] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.285667] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.285701] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.285709] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.285713] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.287640] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.320548] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.326414] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.332474] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.339040] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.345278] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.352085] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.358207] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.365270] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.371173] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.377436] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.384525] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.399069] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.403133] msm_serial: driver initialized gitlab # [ 0.403473] SuperH (H)SCI(F) driver initialized gitlab # [ 0.403618] STM32 USART driver initialized gitlab # [ 0.449283] loop: module loaded gitlab # [ 0.449881] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.455200] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.460103] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.465502] virtio_blk virtio5: [vdb] 12285240 512-byte logical blocks (6.29 GB/5.86 GiB) gitlab # [ 0.470887] megasas: 07.734.00.00-rc1 gitlab # [ 0.472860] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.478820] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.478930] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.484207] Using buffer write method gitlab # [ 0.484275] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.488969] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.489004] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.493813] Using buffer write method gitlab # [ 0.493849] Concatenating MTD devices: gitlab # [ 0.493856] (0): "0.flash" gitlab # [ 0.493862] (1): "0.flash" gitlab # [ 0.493868] into device "0.flash" gitlab # [ 0.652771] Freeing initrd memory: 27184K gitlab # [ 0.665512] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.670528] thunder_xcv, ver 1.0 gitlab # [ 0.670589] thunder_bgx, ver 1.0 gitlab # [ 0.670616] nicpf, ver 1.0 gitlab # [ 0.671194] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.671199] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.671221] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.671227] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.671255] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.671258] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.671281] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.671285] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.671424] sky2: driver version 1.30 gitlab # [ 0.673114] usbcore: registered new interface driver usb-storage gitlab # [ 0.673301] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.673319] usbserial: USB Serial support registered for generic gitlab # [ 0.673956] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.674234] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.674283] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.674842] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.675050] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.675078] rtc-pl031 9010000.pl031: setting system clock to 2026-09-14T08:34:44 UTC (1789374884) gitlab # [ 0.675402] i2c_dev: i2c /dev entries driver gitlab # [ 0.678331] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.678337] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.678608] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.679005] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.680744] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.681045] usbcore: registered new interface driver usbhid gitlab # [ 0.681049] usbhid: USB HID core driver gitlab # [ 0.699624] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.700652] hub 1-0:1.0: USB hub found gitlab # [ 0.700694] hub 1-0:1.0: 6 ports detected gitlab # [ 0.718084] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.720219] drop_monitor: Initializing network drop monitor service gitlab # [ 0.720425] NET: Registered PF_INET6 protocol family gitlab # [ 0.721551] Segment Routing with IPv6 gitlab # [ 0.721569] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.721602] NET: Registered PF_PACKET protocol family gitlab # [ 0.721900] 9pnet: Installing 9P2000 support gitlab # [ 0.724168] Key type dns_resolver registered gitlab # [ 0.732433] registered taskstats version 1 gitlab # [ 0.732721] Loading compiled-in X.509 certificates gitlab # [ 0.743949] Demotion targets for Node 0: null gitlab # [ 0.744144] Key type .fscrypt registered gitlab # [ 0.744149] Key type fscrypt-provisioning registered gitlab # [ 0.744359] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.744380] ima: Allocated hash algorithm: sha1 gitlab # [ 0.744405] ima: No architecture policies found gitlab # [ 0.745261] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.765950] clk: Disabling unused clocks gitlab # [ 0.766023] PM: genpd: Disabling unused power domains gitlab # [ 0.770498] Freeing unused kernel memory: 4736K gitlab # [ 0.770692] Run /init as init process gitlab # [ 0.800811] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.947588] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.104429] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.135197] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.135268] systemd[1]: Detected virtualization qemu. gitlab # [ 1.135345] systemd[1]: Detected architecture arm64. gitlab # [ 1.135362] systemd[1]: Running in initrd. gitlab # [ 1.136327] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.136721] systemd[1]: Hostname set to . gitlab # [ 1.244250] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.363591] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.413315] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.510109] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.517806] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.518445] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.531934] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.532324] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.532375] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.532396] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.532436] systemd[1]: Reached target Path Units. gitlab # [ 1.532463] systemd[1]: Reached target Slice Units. gitlab # [ 1.532491] systemd[1]: Reached target Swaps. gitlab # [ 1.532522] systemd[1]: Reached target Timer Units. gitlab # [ 1.532835] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.533083] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.533342] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.533568] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.533707] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.533740] systemd[1]: Reached target Socket Units. gitlab # [ 1.536335] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.538836] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.538943] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.541519] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.569064] systemd[1]: Starting Journal Service... gitlab # [ 1.571479] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.571671] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.579404] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.583694] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.585876] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.586683] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.587295] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.591101] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.606795] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.607117] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.614623] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.614806] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.614809] [drm] features: -context_init gitlab # [ 1.615671] [drm] number of scanouts: 1 gitlab # [ 1.615690] [drm] number of cap sets: 0 gitlab # [ 1.616418] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.616424] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.620332] systemd-journald[96]: Collecting audit messages is disabled. gitlab # [ 1.623679] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.627720] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.630928] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.635980] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.650405] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.652530] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.663241] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.663971] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.664341] systemd[1]: Reached target Local File Systems. gitlab # [ 1.673467] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.674389] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.700818] systemd[1]: Started Journal Service. gitlab # [ 1.697356] systemd-modules-load[97]: Using 4 probe threads gitlab # [ 1.700918] systemd-modules-load[97]: Module 'virtio_balloon' is built in gitlab # [ 1.702240] systemd-modules-load[97]: Module 'virtio_console' is built in gitlab # [ 1.703516] systemd-modules-load[97]: Module 'virtio_rng' is built in gitlab # [ 1.705325] systemd-modules-load[97]: Inserted module 'dm_mod' gitlab # [ 1.706514] systemd-modules-load[97]: Inserted module 'virtio_gpu' gitlab # [ 1.707670] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.728519] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.734752] systemd-udevd[106]: Using default interface naming scheme 'v261'. gitlab # [ 1.754919] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.777445] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.868728] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.873691] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.219298] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.222095] systemd[1]: Reached target System Initialization. gitlab # [ 2.224679] systemd[1]: Reached target Basic System. gitlab # [ 2.443259] (udev-worker)[128]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.454363] (udev-worker)[122]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.464501] (udev-worker)[122]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.467180] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.473432] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.478187] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.479943] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.528402] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.535341] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.580331] systemd[1]: Mounting /sysroot... gitlab # [ 2.629475] EXT4-fs (vda): mounted filesystem 04bc9a25-9c66-4e7c-80e3-817de474bdbb r/w with ordered data mode. Quota mode: none. gitlab # [ 2.629600] systemd[1]: Mounted /sysroot. gitlab # [ 2.633446] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.639661] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.641517] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.661151] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.668400] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.672961] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.684471] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.710984] systemd-sysroot-fstab-check[166]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.720710] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.723925] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.741967] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.743589] systemd[1]: Reload requested from client PID 166 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.746527] systemd[1]: Reloading... gitlab # [ 2.868837] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 2.935098] systemd[1]: Reloading finished in 190 ms. gitlab # [ 2.992705] systemd-sysroot-fstab-check[166]: Requesting initrd-fs.target/start/replace... gitlab # [ 2.994952] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 2.996964] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.999016] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 2.999788] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.000899] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.003804] systemd-sysroot-fstab-check[166]: Requesting swap.target/start/replace... gitlab # [ 3.006974] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.008021] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.008994] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.010293] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.033021] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.034898] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.072221] (udev-worker)[141]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 3.075708] (udev-worker)[141]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 3.084366] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.085409] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.086162] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 3.086903] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.105026] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.106126] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.106851] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.128872] systemd-vconsole-setup[200]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 3.131032] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.586104] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.655007] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.657418] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.659845] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.662803] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.711431] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.715142] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.727277] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.729787] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.732273] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.784382] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.786898] systemd[1]: Stopped target Basic System. gitlab # [ 3.789153] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.791509] systemd[1]: Stopped target Path Units. gitlab # [ 3.793723] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.796929] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.800190] systemd[1]: Stopped target Slice Units. gitlab # [ 3.802335] systemd[1]: Stopped target Socket Units. gitlab # [ 3.804564] systemd[1]: Stopped target System Initialization. gitlab # [ 3.807008] systemd[1]: Stopped target Swaps. gitlab # [ 3.809023] systemd[1]: Stopped target Timer Units. gitlab # [ 3.811141] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.815102] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.817964] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.821541] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.824472] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.826863] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.829233] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.831629] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.833744] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.836404] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.838262] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.841263] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.843984] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.846574] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.848844] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.850809] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.853275] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.855636] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.857618] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.860000] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.862195] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.863748] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.865715] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.867453] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.870652] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.872229] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.874141] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.875765] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.877598] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.879500] systemd[1]: systemd-udevd.service: Consumed 2.354s CPU time over 2.192s wall clock time, 31.2M memory peak. gitlab # [ 3.882310] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.884216] systemd[1]: Closed udev Control Socket. gitlab # [ 3.885556] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.887082] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.889371] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.891046] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.893234] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.895044] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.896810] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 3.961242] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 3.964524] systemd[1]: Finished Cleanup udev Database. gitlab # [ 3.966809] systemd[1]: Reached target Switch Root. gitlab # [ 3.968991] systemd[1]: Starting NixOS Activation... gitlab # [ 4.138812] initrd-nixos-activation-start[225]: booting system configuration /nix/store/apgr4bbsy6asy7kwdy25hr9g9xymxn1n-nixos-system-gitlab-test gitlab # [ 4.201026] initrd-nixos-activation-start[225]: running activation script... gitlab # [ 4.518525] initrd-nixos-activation-start[248]: setting up /etc... gitlab # [ 4.668322] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 4.671599] systemd[1]: Finished NixOS Activation. gitlab # [ 4.673853] systemd[1]: Starting Switch Root... gitlab # [ 4.721461] systemd[1]: Switching root. gitlab # [ 4.839884] systemd-journald[96]: Received SIGTERM from PID 1 (systemd). gitlab # [ 5.452816] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 5.452964] systemd[1]: Detected virtualization qemu. gitlab # [ 5.453046] systemd[1]: Detected architecture arm64. gitlab # [ 5.453285] systemd[1]: Detected first boot. gitlab # [ 5.454710] systemd[1]: Initializing machine ID from random generator. gitlab # [ 5.704060] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 5.829603] systemd[1]: Applying preset policy. gitlab # [ 6.013026] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.167859] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.168648] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.171166] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.178984] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.182895] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.187062] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.188583] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.189978] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.191130] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.192162] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 6.192511] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.192784] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.193028] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.193274] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.193529] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.193785] systemd[1]: Reached target Path Units. gitlab # [ 6.194024] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.194260] systemd[1]: Reached target Slice Units. gitlab # [ 6.194495] systemd[1]: Reached target Swaps. gitlab # [ 6.195856] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.197240] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.198162] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.199236] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.199605] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.206882] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.208029] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.209847] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.210537] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.211151] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.211870] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.214514] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.215104] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.215867] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 6.227943] systemd[1]: Mounting Huge Pages File System... gitlab # [ 6.236480] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 6.244466] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 6.254481] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 6.267805] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 6.287894] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 6.288990] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 6.299004] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 6.299954] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 6.300900] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 6.309084] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 6.310118] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 6.323672] systemd[1]: Starting Journal Service... gitlab # [ 6.329841] systemd[1]: Starting Load Kernel Modules... gitlab # [ 6.331010] fuse: init (API version 7.45) gitlab # [ 6.339771] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 6.346602] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 6.354760] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.359858] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 6.366783] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 6.367965] systemd[1]: Mounted Huge Pages File System. gitlab # [ 6.368767] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 6.369686] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 6.370670] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 6.371039] systemd-journald[319]: Collecting audit messages is enabled. gitlab # [ 6.375338] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 6.376643] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 6.377771] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 6.378766] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 6.394183] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 6.379885] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 6.380947] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 6.382377] systemd[1]: Finished Load Kernel Modules. gitlab # [ 6.405207] systemd[1]: Unnecessary job was removed for /dev/vdb.[ 6.398928] systemd[1]: Mounting FUSE Control File System... gitlab # gitlab # [ 6.402104] EXT4-fs (vda): re-mounted 04bc9a25-9c66-4e7c-80e3-817de474bdbb. gitlab # [ 6.413787] systemd[1]: systemd-journald.service: Deactivated successfully. gitlab # [ 6.415254] systemd-modules-load[320]: Using 3 probe threads gitlab # [ 6.416399] systemd-modules-load[320]: Module 'atkbd' is built in[ 6.419814] systemd[1]: Starting Firewall... gitlab # gitlab # [ 6.419763] systemd-modules-load[320]: Module 'loop' is built in gitlab # [ 6.421050] systemd-modules-load[320]: Inserted module 'tls' gitlab # [ 6.433845] systemd-oomd[322]: No swap; memory pressure usage will be degraded gitlab # [ 6.450633] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 6.459509] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 6.463961] systemd[1]: Started Journal Service. gitlab # [ 6.462604] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 6.466237] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 6.470152] systemd[1]: Mounted FUSE Control File System. gitlab # [ 6.481849] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 6.487784] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 6.492365] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 6.494876] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 6.500284] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 6.504210] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.517948] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 6.527296] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 6.535275] systemd-journald[319]: Received client request to flush runtime journal. gitlab # [ 6.567447] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 6.569337] systemd[1]: Reached target First Boot Complete. gitlab # [ 6.570338] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 6.571273] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 6.572320] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 6.580785] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 6.633898] systemd-udevd[369]: Using default interface naming scheme 'v261'. gitlab # [ 6.667735] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 6.974889] systemd[1]: Finished Firewall. gitlab # [ 6.989752] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.056517] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.123291] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 7.129548] (udev-worker)[476]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.154940] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 7.156871] (udev-worker)[412]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 7.160149] (udev-worker)[412]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 7.168635] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.190791] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 7.193108] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.195085] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.198164] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.199722] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.201242] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.203263] systemd[1]: Mounted /run/wrappers. gitlab # [ 7.204497] systemd[1]: Reached target Local File Systems. gitlab # [ 7.205791] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 7.207982] systemd[1]: Starting register-nix-paths.service... gitlab # [ 7.211882] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 7.213865] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 7.222927] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 7.232370] systemd[1]: Starting Create System Files and Directories... gitlab # [ 7.259914] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 7.262349] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 7.288619] systemd[1]: Finished Create System Files and Directories. gitlab # [ 7.294344] mousedev: PS/2 mouse device common for all mice gitlab # [ 7.297117] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 7.300338] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 7.332138] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 7.348851] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 7.352190] systemd[1]: Starting Update is Completed... gitlab # [ 7.388751] systemd[1]: Finished Update is Completed. gitlab # [ 7.399983] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 7.403078] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 7.404649] systemd[1]: suid-sgid-wrappers.service: Consumed 137ms CPU time over 189ms wall clock time, 6M memory peak, 1.9M read from disk. gitlab # [ 7.696858] systemd[1]: Finished register-nix-paths.service. gitlab # [ 7.699832] systemd[1]: Reached target System Initialization. gitlab # [ 7.704723] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 7.705846] systemd[1]: Started logrotate.timer. gitlab # [ 7.706559] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 7.707523] systemd[1]: Reached target Timer Units. gitlab # [ 7.708363] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 7.717804] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 7.785652] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 7.793830] systemd[1]: Starting nixbot CI service socket... gitlab # [ 7.798475] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 7.801277] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 7.805762] systemd[1]: Reached target Socket Units. gitlab # [ 7.819898] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 7.831382] systemd[1]: Starting Virtual Console Setup... gitlab # [ 7.858884] dbus-broker-launch[567]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 7.861243] dbus-broker-launch[567]: NSS returned no entry for 'systemd-timesync' gitlab # [ 7.862836] dbus-broker-launch[567]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 7.866464] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 7.875208] dbus-broker-launch[567]: Ready gitlab # [ 7.876279] systemd[1]: Reached target Basic System. gitlab # [ 7.879842] systemd[1]: Started backdoor.service. gitlab # [ 7.881411] systemd[1]: Starting gitlab-config.service... gitlab # [ 7.886582] systemd-vconsole-setup[569]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 7.898139] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 7.905635] systemd[1]: Starting Logrotate configuration check... gitlab # [ 7.908450] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 7.913601] systemd[1]: Starting Post-Boot Actions... gitlab # connecting to host... gitlab # [ 7.918794] systemd[1]: Started Reset console on configuration changes. gitlab # [ 7.921999] logrotate[575]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 7.924533] logrotate[575]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 7.925938] logrotate[575]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 7.927623] logrotate[575]: Reading state from file: /var/lib/logrotate.status gitlab # [ 7.929012] logrotate[575]: state file /var/lib/logrotate.status does not exist gitlab # [ 7.930071] logrotate[575]: Allocating hash table for state file, size 64 entries gitlab # [ 7.931102] logrotate[575]: Handling 4 logs gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 7.931704] logrotate[575]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.934525] logrotate[575]: considering log /var/log/btmp gitlab # [ 7.935345] logrotate[575]: Creating new state gitlab # [ 7.935948] logrotate[575]: Now: 2026-09-14 08:34 gitlab: connected to guest root shell gitlab # [ 7.936708] logrotate[575]: Last rotated at 2026-09-14 08:00 gitlab: (connecting took 8.41 seconds) gitlab # [ 7.937427] logrotate[575]: log does not need rotating (log has already been rotated) gitlab: (finished: waiting for the VM to finish booting, in 8.41 seconds) gitlab # [ 7.938454] logrotate[575]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 7.940544] logrotate[575]: considering log /var/log/wtmp gitlab # [ 7.941261] logrotate[575]: Creating new state gitlab # [ 7.941819] logrotate[575]: Now: 2026-09-14 08:34 gitlab # [ 7.942415] logrotate[575]: Last rotated at 2026-09-14 08:00 gitlab # [ 7.943248] logrotate[575]: log does not need rotating (log has already been rotated) gitlab # [ 7.944447] logrotate[575]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 7.946125] logrotate[575]: switching euid from 0 to 165 and egid from 0 to 165 (pid 575) gitlab # [ 7.947161] logrotate[575]: considering log /var/gitlab/state/log/*.log gitlab # [ 7.948040] logrotate[575]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab # [ 7.949052] logrotate[575]: Creating new state gitlab # [ 7.949608] logrotate[575]: switching euid from 165 to 0 and egid from 165 to 0 (pid 575) gitlab # [ 7.950585] logrotate[575]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 7.952643] logrotate[575]: switching euid from 0 to 60 and egid from 0 to 60 (pid 575) gitlab # [ 7.953620] logrotate[575]: considering log /var/log/nginx/*.log gitlab # [ 7.954499] logrotate[575]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 7.955402] logrotate[575]: Creating new state gitlab # [ 7.955984] logrotate[575]: not running postrotate script, since no logs were rotated gitlab # [ 7.957718] logrotate[575]: switching euid from 60 to 0 and egid from 60 to 0 (pid 575) gitlab # [ 7.959356] systemd[1]: Starting resolvconf update... gitlab # [ 7.960490] nsncd[578]: Sep 14 08:34:51.783 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 7.963650] systemd[1]: Finished Virtual Console Setup. gitlab # [ 7.964603] systemd[1]: Finished Logrotate configuration check. gitlab # [ 7.965546] systemd[1]: Finished Post-Boot Actions. gitlab # [ 7.966329] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 7.967197] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 7.968785] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 7.970291] systemd[1]: Starting User Login Management... gitlab # [ 7.971115] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.015241] systemd-logind[618]: New seat seat0. gitlab # [ 8.018766] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.022090] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.023047] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.023913] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.027577] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.030243] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.031108] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.032480] systemd-logind[618]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.033742] systemd-logind[618]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 8.035020] systemd-logind[618]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 8.037341] systemd[1]: nscd.service: Consumed 22ms CPU time over 114ms wall clock time, 7.3M memory peak, 5.8M read from disk. gitlab # [ 8.038896] systemd[1]: Started User Login Management. gitlab # [ 8.039607] systemd[1]: Starting linger-users.service... gitlab # [ 8.041563] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.062658] systemd[1]: Finished resolvconf update. gitlab # [ 8.065930] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.067203] nsncd[671]: Sep 14 08:34:51.894 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.069503] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.070467] systemd[1]: Finished linger-users.service. gitlab # [ 8.071184] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.072181] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.073063] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.076099] systemd[1]: Starting DHCP Client... gitlab # [ 8.078427] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.082588] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.111487] network-addresses-eth1-start[694]: adding address 192.168.1.1/24... done gitlab # [ 8.118307] network-addresses-eth1-start[694]: adding address 2001:db8:1::1/64... done gitlab # [ 8.129037] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.164558] dhcpcd[706]: dhcpcd-10.3.2 starting gitlab # [ 8.165416] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.166647] dhcpcd[758]: dev: loaded udev gitlab # [ 8.167341] systemd[1]: Reached target Network. gitlab # [ 8.169769] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.173493] systemd[1]: Starting Nginx Web Server... gitlab # [ 8.178820] systemd[1]: Starting PostgreSQL Server... gitlab # [ 8.183029] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 8.186191] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 8.186589] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 8.187006] systemd[1]: Starting Permit User Sessions... gitlab # [ 8.199667] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 8.215739] systemd[1]: Finished Permit User Sessions. gitlab # [ 8.228114] systemd[1]: Started Getty on tty1. gitlab # [ 8.230242] systemd[1]: Reached target Login Prompts. gitlab # [ 8.234227] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 8.242590] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 8.243089] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 8.245008] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 8.245347] cfg80211: failed to load regulatory.db gitlab # [ 8.296889] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # [ 8.294474] dhcpcd[758]: eth0: waiting for carrier gitlab # [ 8.298648] dhcpcd[758]: eth0: carrier acquired gitlab # [ 8.302852] dhcpcd[758]: DUID 00:01:00:01:32:3a:6e:2c:52:54:00:12:34:56 gitlab # [ 8.303847] dhcpcd[758]: eth0: IAID 00:12:34:56 gitlab # [ 8.304761] dhcpcd[758]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 8.334522] nginx-pre-start[790]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 8.336649] nginx-pre-start[790]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 8.341005] systemd[1]: Started Nginx Web Server. gitlab # [ 8.370259] redis-server[788]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 8.372775] redis-server[788]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 8.373883] redis-server[788]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=788, just started gitlab # [ 8.375302] redis-server[788]: Configuration loaded gitlab # [ 8.376263] redis[788]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 8.378673] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 8.379842] redis[788]: Running mode=standalone, port=0. gitlab # [ 8.380992] redis[788]: Server initialized gitlab # [ 8.382083] redis[788]: Ready to accept connections unix gitlab # [ 8.383179] redis[788]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 8.400675] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 8.402213] postgresql-pre-start[794]: This user must also own the server process. gitlab # [ 8.407197] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 8.408565] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 8.410623] postgresql-pre-start[794]: The default text search configuration will be set to "english". gitlab # [ 8.412360] postgresql-pre-start[794]: Data page checksums are enabled. gitlab # [ 8.415258] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 8.416799] postgresql-pre-start[794]: creating subdirectories ... ok gitlab # [ 8.417793] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix gitlab # [ 8.449383] postgresql-pre-start[794]: selecting default "max_connections" ... 100 gitlab # [ 8.499813] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB gitlab # [ 8.795051] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.041711] postgresql-pre-start[794]: selecting default time zone ... UTC gitlab # [ 9.045375] postgresql-pre-start[794]: creating configuration files ... ok gitlab # [ 9.067136] gitlab-workhorse[820]: time="2026-09-14T08:34:52Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 9.068868] gitlab-workhorse[820]: time="2026-09-14T08:34:52Z" level=info msg="Using redis/go-redis" gitlab # [ 9.070116] gitlab-workhorse[820]: time="2026-09-14T08:34:52Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 9.201061] dhcpcd[758]: eth0: soliciting a DHCP lease gitlab # [ 9.209114] dhcpcd[758]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 9.228489] dhcpcd[758]: eth0: probing address 10.0.2.15/24 gitlab # [ 9.285730] postgresql-pre-start[794]: running bootstrap script ... ok gitlab # [ 10.015950] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok gitlab # [ 10.328408] postgresql-pre-start[794]: syncing data to disk ... ok gitlab # [ 10.330746] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 10.334084] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 10.339158] postgresql-pre-start[794]: Success. You can now start the database server using: gitlab # [ 10.341914] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 10.524379] postgres[855]: [855] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 10.530475] postgres[855]: [855] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 10.534898] postgres[855]: [855] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 10.538573] postgres[855]: [855] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 10.543162] postgres[868]: [868] LOG: database system was shut down at 2026-09-14 08:34:53 GMT gitlab # [ 10.547905] postgres[855]: [855] LOG: database system is ready to accept connections gitlab # [ 10.551946] systemd[1]: Started PostgreSQL Server. gitlab # [ 10.554201] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 10.623342] dhcpcd[758]: eth0: soliciting an IPv6 router gitlab # [ 10.627036] dhcpcd[758]: eth0: Router Advertisement from fe80::2 gitlab # [ 10.629839] dhcpcd[758]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 10.634364] dhcpcd[758]: eth0: adding route to fec0::/64 gitlab # [ 10.636748] dhcpcd[758]: eth0: adding default route via fe80::2 gitlab # [ 10.753953] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[862]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 10.758482] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[862]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 10.830657] postgresql-setup-start[880]: CREATE DATABASE gitlab # [ 10.841181] systemd[1]: Finished gitlab-config.service. gitlab # [ 10.847408] systemd[1]: Started gitaly.service. gitlab # [ 10.887279] postgresql-setup-start[898]: CREATE ROLE gitlab # [ 10.909031] postgresql-setup-start[901]: ALTER DATABASE gitlab # [ 10.958808] postgresql-setup-start[906]: CREATE ROLE gitlab # [ 10.961946] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 10.964425] systemd[1]: Reached target PostgreSQL. gitlab # [ 10.966606] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 10.972141] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 11.183163] gitlab-postgresql-start[916]: CREATE DATABASE gitlab # [ 11.232173] gitlab-postgresql-start[935]: CREATE EXTENSION gitlab # [ 11.318564] gitlab-postgresql-start[938]: CREATE EXTENSION gitlab # [ 11.324181] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 11.645067] gitaly[897]: time="2026-09-14T08:34:55.472Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=897 gitlab # [ 11.656631] gitaly[897]: time="2026-09-14T08:34:55.473Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=897 gitlab # [ 11.664205] gitaly[897]: time="2026-09-14T08:34:55.473Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=897 gitlab # [ 11.669037] gitaly[897]: time="2026-09-14T08:34:55.473Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=897 gitlab # [ 11.673146] gitaly[897]: time="2026-09-14T08:34:55.473Z" level=info msg="Starting Gitaly" pid=897 version= gitlab # [ 11.890666] gitaly[897]: time="2026-09-14T08:34:55.718Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=244 pid=897 gitlab # [ 11.893204] gitaly[897]: time="2026-09-14T08:34:55.718Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=897 gitlab # [ 11.895545] gitaly[897]: time="2026-09-14T08:34:55.719Z" level=info msg="finished initializing command factory" duration_ms=0 pid=897 gitlab # [ 11.898182] gitaly[897]: time="2026-09-14T08:34:55.719Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-897/git-exec-3187013438.d/git pid=897 gitlab # [ 11.903213] gitaly[897]: time="2026-09-14T08:34:55.731Z" level=info msg="finished detecting git version" duration_ms=12 pid=897 gitlab # [ 11.905541] gitaly[897]: time="2026-09-14T08:34:55.731Z" level=info msg="using Git version" pid=897 version=2.53.g gitlab # [ 11.907719] gitaly[897]: time="2026-09-14T08:34:55.731Z" level=info msg="clearing disk cache object folder" pid=897 storage=default gitlab # [ 11.909989] gitaly[897]: time="2026-09-14T08:34:55.732Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2637128061 pid=897 storage=default gitlab # [ 11.913799] gitaly[897]: time="2026-09-14T08:34:55.732Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2637128061 pid=897 storage=default gitlab # [ 11.918493] gitaly[897]: time="2026-09-14T08:34:55.732Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=897 gitlab # [ 11.921607] gitaly[897]: time="2026-09-14T08:34:55.732Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=897 gitlab # [ 11.924762] gitaly[897]: time="2026-09-14T08:34:55.732Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2637128061 clear_duration_ms=0 pid=897 storage=default gitlab # [ 11.930421] gitaly[897]: time="2026-09-14T08:34:55.758Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=897 gitlab # [ 11.933637] gitaly[897]: time="2026-09-14T08:34:55.758Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=897 schema=unix gitlab # [ 11.936774] gitaly[897]: time="2026-09-14T08:34:55.759Z" level=info msg="listening at address" address=/run/gitaly/gitaly-897/sock.d/intern pid=897 schema=unix gitlab # [ 11.939590] gitaly[897]: time="2026-09-14T08:34:55.759Z" level=info msg="counting repositories" pid=897 gitlab # [ 11.941670] gitaly[897]: time="2026-09-14T08:34:55.760Z" level=info msg="starting to count repositories" pid=897 storage_path=/var/gitlab/state/repositories gitlab # [ 11.944692] gitaly[897]: time="2026-09-14T08:34:55.760Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=897 gitlab # [ 11.947180] gitaly[897]: time="2026-09-14T08:34:55.761Z" level=info msg="Started Gitaly" duration_ms=287 pid=897 gitlab # [ 11.949449] gitaly[897]: time="2026-09-14T08:34:55.760Z" level=info msg="finished tempdir cleaner walk" pid=897 storage=default time_ms=1 gitlab # [ 11.952117] gitaly[897]: time="2026-09-14T08:34:55.761Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=897 gitlab # [ 11.954718] gitaly[897]: time="2026-09-14T08:34:55.761Z" level=info msg="maintenance: daily scheduled" pid=897 scheduled="2026-09-14 12:00:00 +0000 UTC" gitlab # [ 11.957771] gitaly[897]: time="2026-09-14T08:34:55.761Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=897 storage_path=/var/gitlab/state/repositories gitlab # [ 11.961442] gitaly[897]: time="2026-09-14T08:34:55.762Z" level=info msg="completed counting all repositories" counting_duration=1 pid=897 gitlab # [ 12.283058] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.287404] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 12.307267] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 14.987385] dhcpcd[758]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 14.987681] dhcpcd[758]: eth0: adding route to 10.0.2.0/24 gitlab # [ 14.987811] dhcpcd[758]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.178222] systemd[1]: Started DHCP Client. gitlab # [ 15.180359] systemd[1]: Reached target Network is Online. gitlab # [ 15.398641] gitaly[897]: time="2026-09-14T08:34:59.225Z" level=info msg="License database preloaded" duration_ms=3752 pid=897 gitlab # [ 57.865008] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: W, [2026-09-14T08:35:41.691525 #911] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 59.157501] gitlab-workhorse[820]: time="2026-09-14T08:35:42Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 60.907254] postgres[1442]: [1442] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 60.907692] postgres[1442]: [1442] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 60.907851] postgres[1442]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 60.907974] postgres[1442]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 60.908192] postgres[1442]: attidentity AS identity, gitlab # [ 60.908312] postgres[1442]: attgenerated as attgenerated gitlab # [ 60.908430] postgres[1442]: FROM pg_attribute a gitlab # [ 60.908549] postgres[1442]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 60.908668] postgres[1442]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 60.908797] postgres[1442]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 60.908914] postgres[1442]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 60.909055] postgres[1442]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 60.909196] postgres[1442]: ORDER BY a.attnum gitlab # [ 60.909297] postgres[1442]: gitlab # [ 65.968751] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1493]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 65.969252] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1493]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 69.163124] gitlab-workhorse[820]: time="2026-09-14T08:35:52Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 79.171741] gitlab-workhorse[820]: time="2026-09-14T08:36:02Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 89.178059] gitlab-workhorse[820]: time="2026-09-14T08:36:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 92.952510] postgres[1605]: [1605] ERROR: relation "table_without_models" does not exist gitlab # [ 92.952923] postgres[1605]: [1605] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 99.189087] gitlab-workhorse[820]: time="2026-09-14T08:36:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 109.198324] gitlab-workhorse[820]: time="2026-09-14T08:36:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 119.204525] gitlab-workhorse[820]: time="2026-09-14T08:36:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 123.079568] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 123.085357] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 123.090679] systemd[1]: Starting gitlab.service... gitlab # [ 123.604269] bundle[1980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 123.610407] bundle[1980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 123.616341] bundle[1980]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 128.244998] sidekiq-cluster[1979]: {"severity":"INFO","time":"2026-09-14T08:36:52.045Z","message":"Starting cluster with 1 processes"} gitlab # [ 128.280878] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.076Z","pid":1980,"message":"* Enabling systemd notification integration"} gitlab # [ 128.282833] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"Puma starting in cluster mode..."} gitlab # [ 128.285684] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 128.287560] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 128.290017] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Min threads: 0"} gitlab # [ 128.291565] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Max threads: 4"} gitlab # [ 128.293177] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Environment: production"} gitlab # [ 128.294763] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Master PID: 1980"} gitlab # [ 128.296342] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Workers: 2"} gitlab # [ 128.297829] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 128.299735] bundle[1980]: {"timestamp":"2026-09-14T08:36:52.104Z","pid":1980,"message":"* Preloading application"} gitlab # [ 129.218810] gitlab-workhorse[820]: time="2026-09-14T08:36:53Z" level=error correlation_id= duration_ms=1 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 139.222789] gitlab-workhorse[820]: time="2026-09-14T08:37:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 149.223979] gitlab-workhorse[820]: time="2026-09-14T08:37:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 159.225304] gitlab-workhorse[820]: time="2026-09-14T08:37:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 165.635308] bundle[1980]: W, [2026-09-14T08:37:29.462390 #1980] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 169.226473] gitlab-workhorse[820]: time="2026-09-14T08:37:33Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 170.221942] sidekiq-cluster[2020]: 2026-09-14T08:37:34.048Z pid=2020 tid=wng INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2020, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>428490752} gitlab # [ 170.494077] sidekiq-cluster[2020]: W, [2026-09-14T08:37:34.322139 #2020] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 170.640770] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:34.468Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 174.890493] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.717Z","message":"GitLab reliable fetch activated!"} gitlab # [ 174.890912] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.717Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 174.944391] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.772Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 174.956733] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.784Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 174.978927] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.806Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 174.987816] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.815Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 174.995453] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.823Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 175.002566] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.830Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 175.009476] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.835Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 175.015390] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.839Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 175.020767] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.842Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 175.025902] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.846Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 175.031144] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.849Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 175.035804] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.853Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 175.039863] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.857Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 175.043981] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.860Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 175.048598] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.863Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 175.052269] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.866Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 175.055727] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.871Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 175.059236] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.875Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 175.063072] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.879Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 175.066739] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.883Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 175.070734] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.887Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 175.074798] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.890Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 175.078185] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.894Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 175.081496] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.897Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 175.084746] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.899Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 175.087645] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.903Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 175.090526] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.906Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 175.093339] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.909Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 175.096077] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.912Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 175.098804] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.916Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 175.101363] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.918Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 175.104067] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.921Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 175.106401] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.924Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 175.108970] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.927Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 175.111311] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.929Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 175.113733] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.932Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 175.116023] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.934Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 175.118208] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.936Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 175.120473] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.938Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 175.122586] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.940Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 175.125031] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.943Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 175.127161] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.945Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 175.129473] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.947Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 175.131577] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.949Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 175.133918] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.952Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 175.136233] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.955Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 175.138389] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.957Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 175.140576] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.959Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 175.142725] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.962Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 175.144919] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.965Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 175.147039] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.967Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 175.149237] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.971Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 175.157303] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.985Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 175.160788] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.989Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 175.163510] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.991Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 175.166366] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.994Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 175.169224] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:38.997Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 175.172452] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.000Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 175.175011] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.002Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 175.177697] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.005Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 175.179986] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.007Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 175.182205] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.009Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 175.262425] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.090Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 175.265626] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.093Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 175.268557] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.096Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 175.272546] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.100Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 175.276658] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.102Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 175.281104] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.105Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 175.285778] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.107Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 175.290516] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.110Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 175.295181] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.112Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 175.299601] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.114Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 175.303795] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.117Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 175.308055] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.119Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 175.311792] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.122Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 175.315592] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.125Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 175.318993] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.127Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 175.322610] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.130Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 175.325690] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.132Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 175.328543] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.135Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 175.331574] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.137Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 175.334765] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.139Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 175.337670] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.142Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 175.340344] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.145Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 175.343017] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.147Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 175.345617] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.150Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 175.348234] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.152Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 175.350636] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.155Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 175.353148] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.157Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 175.355512] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.160Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 175.358045] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.162Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 175.360402] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.163Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 175.362684] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.167Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 175.365182] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.169Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 175.367142] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.172Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 175.369554] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.175Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 175.371721] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.178Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 175.373993] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.179Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 175.376142] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.194Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 175.378246] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.197Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 175.380348] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.199Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 175.382542] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.202Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 175.384612] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.206Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 175.386734] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.208Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 175.389095] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.212Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 175.391383] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.214Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 175.393544] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.216Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 175.395745] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.219Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 175.398103] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:37:39.222Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 179.234199] gitlab-workhorse[820]: time="2026-09-14T08:37:43Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 189.235441] gitlab-workhorse[820]: time="2026-09-14T08:37:53Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 199.246326] gitlab-workhorse[820]: time="2026-09-14T08:38:03Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 209.257412] gitlab-workhorse[820]: time="2026-09-14T08:38:13Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 214.129366] postgres[2739]: [2739] ERROR: relation "table_without_models" does not exist gitlab # [ 214.129683] postgres[2739]: [2739] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 216.687267] sidekiq-cluster[2020]: InitializerConnections Query: SET client_min_messages TO 'warning' gitlab # [ 216.688438] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.692939] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.696851] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.700983] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.704664] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.708352] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.713029] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.718964] sidekiq-cluster[2020]: InitializerConnections Query: SET standard_conforming_strings = on gitlab # [ 216.722751] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.727172] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.731033] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.734978] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.738583] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.742009] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.746293] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.750408] sidekiq-cluster[2020]: InitializerConnections Query: SET intervalstyle = iso_8601 gitlab # [ 216.753175] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.756511] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.759194] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.762132] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.764613] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.766898] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.769851] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.772782] sidekiq-cluster[2020]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT t.oid, t.typname gitlab # [ 216.775840] sidekiq-cluster[2020]: FROM pg_type as t gitlab # [ 216.777073] sidekiq-cluster[2020]: WHERE t.typname IN ('int2', 'int4', 'int8', 'oid', 'float4', 'float8', 'numeric', 'bool', 'timestamp', 'timestamptz') gitlab # [ 216.779828] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.782148] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.784049] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.785988] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.787776] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.789648] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.791844] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.794122] sidekiq-cluster[2020]: InitializerConnections Query: SET SESSION timezone TO 'UTC' gitlab # [ 216.795619] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.797608] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.799205] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.800979] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.802478] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.804015] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.805864] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.807793] sidekiq-cluster[2020]: InitializerConnections Query: /*application:sidekiq,db_config_database:gitlab,db_config_name:main*/ SELECT "feature_gates"."key", "feature_gates"."value" FROM "feature_gates" WHERE "feature_gates"."feature_key" = 'sidekiq_throttling_middleware' gitlab # [ 216.811731] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:359:in `block in current_feature_value' gitlab # [ 216.813513] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:374:in `with_feature' gitlab # [ 216.814905] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:355:in `current_feature_value' gitlab # [ 216.816447] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:141:in `enabled?' gitlab # [ 216.817744] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/feature.rb:156:in `disabled?' gitlab # [ 216.819038] sidekiq-cluster[2020]: InitializerConnections Backtrace: lib/gitlab/background_task.rb:60:in `block (2 levels) in start' gitlab # [ 216.820732] sidekiq-cluster[2020]: See https://docs.gitlab.com/ee/development/rails_initializers.html#database-connections-in-initializers gitlab # [ 216.822821] postgres[2769]: [2769] ERROR: relation "table_without_models" does not exist gitlab # [ 216.824025] postgres[2769]: [2769] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 219.201286] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:23.025Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 219.210316] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:23.037Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 219.216556] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:23.037Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 219.221865] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:23.037Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 219.227920] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:23.039Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 219.262186] gitlab-workhorse[820]: time="2026-09-14T08:38:23Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 221.901698] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:25.728Z","message":"Cleaning working queues"} gitlab # [ 221.903973] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:25.730Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a69b0053b19a253e3e0a4ded","created_at":"2026-09-14T08:36:46.413Z","trace_propagation_headers":{"sentry-trace":"4933ff0096514a689f1ccbe100698d7a-5fa4336c7b7e46aa","baggage":"sentry-trace_id=4933ff0096514a689f1ccbe100698d7a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b6252fe8fb0dce34841618cdff01b8c1","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T08:36:46.424Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a69b0053b19a253e3e0a4ded: start","job_status":"start","queue_duration_s":99.306013,"scheduling_latency_s":99.306075} gitlab # [ 222.271672] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:26.098Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"a69b0053b19a253e3e0a4ded","created_at":"2026-09-14T08:36:46.413Z","trace_propagation_headers":{"sentry-trace":"4933ff0096514a689f1ccbe100698d7a-5fa4336c7b7e46aa","baggage":"sentry-trace_id=4933ff0096514a689f1ccbe100698d7a,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"b6252fe8fb0dce34841618cdff01b8c1","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T08:36:46.424Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-a69b0053b19a253e3e0a4ded: done: 0.368244 sec","job_status":"done","queue_duration_s":99.306013,"scheduling_latency_s":99.306075,"redis_calls":29,"redis_duration_s":0.011911,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007091,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002922,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001898,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.02,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.177984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.18610122,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.368244,"completed_at":"2026-09-14T08:38:26.098Z","load_balancing_strategy":"primary","db_duration_s":0.103055,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 222.391690] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 222.394920] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 222.397787] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"! #\u003cThread:0x0000f59d229507a8 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 222.406536] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"! #\u003cThread:0x0000f59d22b8d3b8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 222.415285] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"Use Ctrl-C to stop"} gitlab # [ 222.417499] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.183Z","pid":1980,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 222.970654] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.798Z","pid":1980,"message":"- Worker 0 (PID: 2878) booted in 0.58s, phase: 0"} gitlab # [ 222.976430] bundle[1980]: {"timestamp":"2026-09-14T08:38:26.801Z","pid":1980,"message":"- Worker 1 (PID: 2882) booted in 0.56s, phase: 0"} gitlab # [ 222.982136] systemd[1]: Started gitlab.service. gitlab # [ 222.984380] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 222.987357] systemd[1]: Starting setup-gitlab.service... gitlab # [ 223.024331] setup-gitlab-start[2915]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 226.662413] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:38:30 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3595 gitlab # [ 226.667397] setup-gitlab-start[2915]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 226.668100] setup-gitlab-start[2915]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 226.687121] runuser[2937]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 227.244168] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 227.250561] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 227.257005] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 228.066681] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 228.067032] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 228.069367] setup-gitlab-start[2938]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 232.181183] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.007Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f5612adb756a8d8ad367cc68","created_at":"2026-09-14T08:38:35.898Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.003Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f5612adb756a8d8ad367cc68: start","job_status":"start","queue_duration_s":0.003585,"scheduling_latency_s":0.003655} gitlab # [ 232.835374] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"f5612adb756a8d8ad367cc68","created_at":"2026-09-14T08:38:35.898Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.003Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-f5612adb756a8d8ad367cc68: done: 0.655045 sec","job_status":"done","queue_duration_s":0.003585,"scheduling_latency_s":0.003655,"redis_calls":71,"redis_duration_s":0.187002,"redis_read_bytes":120,"redis_write_bytes":42624,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.014156,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.002389,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.166302,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004155,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.2756,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00314846,"duration_s":0.655045,"completed_at":"2026-09-14T08:38:36.662Z","load_balancing_strategy":"primary","db_duration_s":0.045627,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 232.930199] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.745Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"90a0a19c97ed2586451de7c7","created_at":"2026-09-14T08:38:36.017Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21875c38ed8312cb150547cf83804da4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:38:36.034Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-90a0a19c97ed2586451de7c7: start","job_status":"start","queue_duration_s":0.710315,"scheduling_latency_s":0.710369} gitlab # [ 233.012354] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.840Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 233.085484] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"90a0a19c97ed2586451de7c7","created_at":"2026-09-14T08:38:36.017Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"21875c38ed8312cb150547cf83804da4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:38:36.034Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-90a0a19c97ed2586451de7c7: done: 0.167349 sec","job_status":"done","queue_duration_s":0.710315,"scheduling_latency_s":0.710369,"redis_calls":10,"redis_duration_s":0.051421,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002705,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.04602,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002696,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.054,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03144332,"duration_s":0.167349,"completed_at":"2026-09-14T08:38:36.912Z","load_balancing_strategy":"primary","db_duration_s":0.053857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 233.176769] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:36.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ab021577b5f7998e39a114a5","created_at":"2026-09-14T08:38:36.062Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e949d4cf386ce06bb64fa150b1b2c779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.078Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ab021577b5f7998e39a114a5: start","job_status":"start","queue_duration_s":0.857616,"scheduling_latency_s":0.857673} gitlab # [ 233.710337] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"ab021577b5f7998e39a114a5","created_at":"2026-09-14T08:38:36.062Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e949d4cf386ce06bb64fa150b1b2c779","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.078Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-ab021577b5f7998e39a114a5: done: 0.601337 sec","job_status":"done","queue_duration_s":0.857616,"scheduling_latency_s":0.857673,"redis_calls":14,"redis_duration_s":0.051047999999999996,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.042163,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006361,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002524,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.059,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.059,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.335107,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00107878,"exclusive_lock_hold_duration_s":0.36744016,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.601337,"completed_at":"2026-09-14T08:38:37.536Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.096749,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 233.807751] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"78d3b13633b108de3ac90f8e","created_at":"2026-09-14T08:38:36.101Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab690fd2f6d3acf2f67ffc9d3ad7b6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.116Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-78d3b13633b108de3ac90f8e: start","job_status":"start","queue_duration_s":1.422626,"scheduling_latency_s":1.422653} gitlab # [ 233.825682] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.566Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"78d3b13633b108de3ac90f8e","created_at":"2026-09-14T08:38:36.101Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab690fd2f6d3acf2f67ffc9d3ad7b6b5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.116Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-78d3b13633b108de3ac90f8e: done: 0.028068 sec","job_status":"done","queue_duration_s":1.422626,"scheduling_latency_s":1.422653,"redis_calls":12,"redis_duration_s":0.0033239999999999997,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001236,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001537,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000551,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000375,"duration_s":0.028068,"completed_at":"2026-09-14T08:38:37.566Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00146,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 233.870741] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.568Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e52310bfa64338f851d4c8e2","created_at":"2026-09-14T08:38:36.123Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9c8f07363e5743f8265dd682e74e4f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.140Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e52310bfa64338f851d4c8e2: start","job_status":"start","queue_duration_s":1.428068,"scheduling_latency_s":1.428093} gitlab # [ 233.885231] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.583Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 233.888768] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.591Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"e52310bfa64338f851d4c8e2","created_at":"2026-09-14T08:38:36.123Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b9c8f07363e5743f8265dd682e74e4f2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.140Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-e52310bfa64338f851d4c8e2: done: 0.022987 sec","job_status":"done","queue_duration_s":1.428068,"scheduling_latency_s":1.428093,"redis_calls":10,"redis_duration_s":0.0027199999999999998,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000583,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001597,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00054,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012902,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033924,"duration_s":0.022987,"completed_at":"2026-09-14T08:38:37.591Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000804,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 233.931010] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.593Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3bdd01c31ebce4c52b363059","created_at":"2026-09-14T08:38:36.163Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddd6799652730bff8542e2e0965d4ada","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.178Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3bdd01c31ebce4c52b363059: start","job_status":"start","queue_duration_s":1.414337,"scheduling_latency_s":1.41436} gitlab # [ 233.945071] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"3bdd01c31ebce4c52b363059","created_at":"2026-09-14T08:38:36.163Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ddd6799652730bff8542e2e0965d4ada","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.178Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-3bdd01c31ebce4c52b363059: done: 0.047413 sec","job_status":"done","queue_duration_s":1.414337,"scheduling_latency_s":1.41436,"redis_calls":16,"redis_duration_s":0.004452,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002536,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001421,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000495,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022544,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052662,"duration_s":0.047413,"completed_at":"2026-09-14T08:38:37.640Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01572,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 233.986976] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.642Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f84b6e8d415dcef99204419e","created_at":"2026-09-14T08:38:36.238Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c82e3ba1e22f3b0cfc69c7463b4c4b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:38:36.257Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f84b6e8d415dcef99204419e: start","job_status":"start","queue_duration_s":1.384888,"scheduling_latency_s":1.384912} gitlab # [ 234.000266] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.680Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f84b6e8d415dcef99204419e","created_at":"2026-09-14T08:38:36.238Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2c82e3ba1e22f3b0cfc69c7463b4c4b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:38:36.257Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f84b6e8d415dcef99204419e: done: 0.037518 sec","job_status":"done","queue_duration_s":1.384888,"scheduling_latency_s":1.384912,"redis_calls":10,"redis_duration_s":0.002893,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001693,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021804,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003791,"duration_s":0.037518,"completed_at":"2026-09-14T08:38:37.679Z","load_balancing_strategy":"primary","db_duration_s":0.007478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.040654] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6cc4b074056f1057d6307e2d","created_at":"2026-09-14T08:38:36.282Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f95ab4322ff9a97ebb3d340f814f590","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.302Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6cc4b074056f1057d6307e2d: start","job_status":"start","queue_duration_s":1.378881,"scheduling_latency_s":1.378906} gitlab # [ 234.054446] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.737Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"6cc4b074056f1057d6307e2d","created_at":"2026-09-14T08:38:36.282Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f95ab4322ff9a97ebb3d340f814f590","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.302Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-6cc4b074056f1057d6307e2d: done: 0.055932 sec","job_status":"done","queue_duration_s":1.378881,"scheduling_latency_s":1.378906,"redis_calls":10,"redis_duration_s":0.002989,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000567,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00182,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000602,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00050644,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.055932,"completed_at":"2026-09-14T08:38:37.737Z","load_balancing_strategy":"primary","db_duration_s":0.029932,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.098470] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.740Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1fc46176e227ec3b8dbe93d9","created_at":"2026-09-14T08:38:36.328Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5ed33fa7bf26523db2e12840e0d7f2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.351Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1fc46176e227ec3b8dbe93d9: start","job_status":"start","queue_duration_s":1.388805,"scheduling_latency_s":1.388826} gitlab # [ 234.111740] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.769Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"1fc46176e227ec3b8dbe93d9","created_at":"2026-09-14T08:38:36.328Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d5ed33fa7bf26523db2e12840e0d7f2","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.351Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-1fc46176e227ec3b8dbe93d9: done: 0.028867 sec","job_status":"done","queue_duration_s":1.388805,"scheduling_latency_s":1.388826,"redis_calls":12,"redis_duration_s":0.004102,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001004,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00068,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018289,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00057774,"duration_s":0.028867,"completed_at":"2026-09-14T08:38:37.769Z","load_balancing_strategy":"primary","db_duration_s":0.000775,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.153643] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.770Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a04ad7bcaf76592dda427dc1","created_at":"2026-09-14T08:38:36.380Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"693eafee931c5015c8f1a4951ea09df8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:38:36.403Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a04ad7bcaf76592dda427dc1: start","job_status":"start","queue_duration_s":1.366948,"scheduling_latency_s":1.366969} gitlab # [ 234.166883] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.786Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 234.170660] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.794Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a04ad7bcaf76592dda427dc1","created_at":"2026-09-14T08:38:36.380Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"693eafee931c5015c8f1a4951ea09df8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:38:36.403Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a04ad7bcaf76592dda427dc1: done: 0.023121 sec","job_status":"done","queue_duration_s":1.366948,"scheduling_latency_s":1.366969,"redis_calls":10,"redis_duration_s":0.0028610000000000003,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000555,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001747,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000559,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034172,"duration_s":0.023121,"completed_at":"2026-09-14T08:38:37.793Z","load_balancing_strategy":"primary","db_duration_s":0.000737,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.211575] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.795Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d53d39f9fc16f2275546277f","created_at":"2026-09-14T08:38:36.430Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"904f61cfeb1dff5f944ef64d4a440730","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.454Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d53d39f9fc16f2275546277f: start","job_status":"start","queue_duration_s":1.340853,"scheduling_latency_s":1.340872} gitlab # [ 234.225922] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.808Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 234.229020] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.816Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"d53d39f9fc16f2275546277f","created_at":"2026-09-14T08:38:36.430Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"904f61cfeb1dff5f944ef64d4a440730","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.454Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-d53d39f9fc16f2275546277f: done: 0.021108 sec","job_status":"done","queue_duration_s":1.340853,"scheduling_latency_s":1.340872,"redis_calls":10,"redis_duration_s":0.0025180000000000003,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000559,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000532,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031344,"duration_s":0.021108,"completed_at":"2026-09-14T08:38:37.816Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000619,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.272468] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.818Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b09767d6476e2be62d0f2e7","created_at":"2026-09-14T08:38:36.481Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.505Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b09767d6476e2be62d0f2e7: start","job_status":"start","queue_duration_s":1.312932,"scheduling_latency_s":1.31295} gitlab # [ 234.285739] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.857Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"8b09767d6476e2be62d0f2e7","created_at":"2026-09-14T08:38:36.481Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.505Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-8b09767d6476e2be62d0f2e7: done: 0.038976 sec","job_status":"done","queue_duration_s":1.312932,"scheduling_latency_s":1.31295,"redis_calls":19,"redis_duration_s":0.005151,"redis_read_bytes":22,"redis_write_bytes":4781,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00122,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000519,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3011,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002607,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000805,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00052016,"duration_s":0.038976,"completed_at":"2026-09-14T08:38:37.857Z","load_balancing_strategy":"primary","db_duration_s":0.003599,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.327699] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4054a5f803636d6c7b9c5398","created_at":"2026-09-14T08:38:36.525Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb60ed27bca7c4c2992d26ddd5809cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.548Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4054a5f803636d6c7b9c5398: start","job_status":"start","queue_duration_s":1.310316,"scheduling_latency_s":1.310334} gitlab # [ 234.341907] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.886Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"4054a5f803636d6c7b9c5398","created_at":"2026-09-14T08:38:36.525Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eb60ed27bca7c4c2992d26ddd5809cd7","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.548Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-4054a5f803636d6c7b9c5398: done: 0.027445 sec","job_status":"done","queue_duration_s":1.310316,"scheduling_latency_s":1.310334,"redis_calls":10,"redis_duration_s":0.002677,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000532,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001554,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014374,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036606,"duration_s":0.027445,"completed_at":"2026-09-14T08:38:37.886Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004882,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.383406] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ff01be05c2db5148a9c03d89","created_at":"2026-09-14T08:38:36.569Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7414cc54f5a260aea16808c6616df531","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.594Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ff01be05c2db5148a9c03d89: start","job_status":"start","queue_duration_s":1.29405,"scheduling_latency_s":1.29407} gitlab # [ 234.397769] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"ff01be05c2db5148a9c03d89","created_at":"2026-09-14T08:38:36.569Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7414cc54f5a260aea16808c6616df531","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.594Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-ff01be05c2db5148a9c03d89: done: 0.02615 sec","job_status":"done","queue_duration_s":1.29405,"scheduling_latency_s":1.29407,"redis_calls":12,"redis_duration_s":0.003506,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000642,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001635,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014075,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058014,"exclusive_lock_hold_duration_s":0.004534300000000001,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.02615,"completed_at":"2026-09-14T08:38:37.914Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.441096] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60f1b68097505aec89e0266a","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60f1b68097505aec89e0266a: start","job_status":"start","queue_duration_s":1.292599,"scheduling_latency_s":1.292618} gitlab # [ 234.453370] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.939Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"60f1b68097505aec89e0266a","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-60f1b68097505aec89e0266a: done: 0.0234 sec","job_status":"done","queue_duration_s":1.292599,"scheduling_latency_s":1.292618,"redis_calls":16,"redis_duration_s":0.004213,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000505,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002576,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003192,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0234,"completed_at":"2026-09-14T08:38:37.939Z","load_balancing_strategy":"primary","db_duration_s":0.000655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.494669] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624fa7cc0ee5f7017d12bbce","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624fa7cc0ee5f7017d12bbce: start","job_status":"start","queue_duration_s":1.317956,"scheduling_latency_s":1.317976} gitlab # [ 234.507098] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.957Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"624fa7cc0ee5f7017d12bbce","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-624fa7cc0ee5f7017d12bbce: done: 0.016576 sec","job_status":"done","queue_duration_s":1.317956,"scheduling_latency_s":1.317976,"redis_calls":13,"redis_duration_s":0.002824,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00186,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010376,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016576,"completed_at":"2026-09-14T08:38:37.957Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.545858] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.959Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f0a696a24db650558ce87b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f0a696a24db650558ce87b: start","job_status":"start","queue_duration_s":1.335948,"scheduling_latency_s":1.335966} gitlab # [ 234.558112] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7f0a696a24db650558ce87b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7f0a696a24db650558ce87b: done: 0.015801 sec","job_status":"done","queue_duration_s":1.335948,"scheduling_latency_s":1.335966,"redis_calls":13,"redis_duration_s":0.002734,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000939,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001795,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009894,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015801,"completed_at":"2026-09-14T08:38:37.974Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.596754] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.976Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c28200c0e978437489cd1e0d","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c28200c0e978437489cd1e0d: start","job_status":"start","queue_duration_s":1.353421,"scheduling_latency_s":1.353439} gitlab # [ 234.608939] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c28200c0e978437489cd1e0d","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c28200c0e978437489cd1e0d: done: 0.015909 sec","job_status":"done","queue_duration_s":1.353421,"scheduling_latency_s":1.353439,"redis_calls":13,"redis_duration_s":0.002698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001779,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010094,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015909,"completed_at":"2026-09-14T08:38:37.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.647166] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:37.994Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de47251f914c3f0faf80ee0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de47251f914c3f0faf80ee0: start","job_status":"start","queue_duration_s":1.370924,"scheduling_latency_s":1.370944} gitlab # [ 234.659396] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.010Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2de47251f914c3f0faf80ee0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2de47251f914c3f0faf80ee0: done: 0.016152 sec","job_status":"done","queue_duration_s":1.370924,"scheduling_latency_s":1.370944,"redis_calls":13,"redis_duration_s":0.003005,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001148,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010398,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016152,"completed_at":"2026-09-14T08:38:38.010Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.699045] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.011Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25b42f0fd5d22f2f8ceba068","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25b42f0fd5d22f2f8ceba068: start","job_status":"start","queue_duration_s":1.388557,"scheduling_latency_s":1.388574} gitlab # [ 234.711778] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25b42f0fd5d22f2f8ceba068","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25b42f0fd5d22f2f8ceba068: done: 0.015756 sec","job_status":"done","queue_duration_s":1.388557,"scheduling_latency_s":1.388574,"redis_calls":13,"redis_duration_s":0.002787,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000955,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001832,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015756,"completed_at":"2026-09-14T08:38:38.027Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.750203] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee07c667ec42e25387c5e155","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee07c667ec42e25387c5e155: start","job_status":"start","queue_duration_s":1.405915,"scheduling_latency_s":1.405934} gitlab # [ 234.762171] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee07c667ec42e25387c5e155","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee07c667ec42e25387c5e155: done: 0.016336 sec","job_status":"done","queue_duration_s":1.405915,"scheduling_latency_s":1.405934,"redis_calls":13,"redis_duration_s":0.0030239999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001255,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001769,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016336,"completed_at":"2026-09-14T08:38:38.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.800575] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd0659d377a7b5cff011dedd","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd0659d377a7b5cff011dedd: start","job_status":"start","queue_duration_s":1.423677,"scheduling_latency_s":1.423693} gitlab # [ 234.812697] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cd0659d377a7b5cff011dedd","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cd0659d377a7b5cff011dedd: done: 0.015561 sec","job_status":"done","queue_duration_s":1.423677,"scheduling_latency_s":1.423693,"redis_calls":13,"redis_duration_s":0.002558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00161,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015561,"completed_at":"2026-09-14T08:38:38.062Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.850656] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3dbbba2a924d2670b92601","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3dbbba2a924d2670b92601: start","job_status":"start","queue_duration_s":1.440638,"scheduling_latency_s":1.440656} gitlab # [ 234.862850] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.079Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f3dbbba2a924d2670b92601","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f3dbbba2a924d2670b92601: done: 0.015809 sec","job_status":"done","queue_duration_s":1.440638,"scheduling_latency_s":1.440656,"redis_calls":13,"redis_duration_s":0.002687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00175,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015809,"completed_at":"2026-09-14T08:38:38.079Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.902491] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbc8fe40b3a6ffc18cb713ce","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbc8fe40b3a6ffc18cb713ce: start","job_status":"start","queue_duration_s":1.457893,"scheduling_latency_s":1.457909} gitlab # [ 234.914532] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cbc8fe40b3a6ffc18cb713ce","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cbc8fe40b3a6ffc18cb713ce: done: 0.016673 sec","job_status":"done","queue_duration_s":1.457893,"scheduling_latency_s":1.457909,"redis_calls":13,"redis_duration_s":0.002888,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000962,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016673,"completed_at":"2026-09-14T08:38:38.097Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 234.952677] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.099Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feec82a5314e7865d44025c6","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feec82a5314e7865d44025c6: start","job_status":"start","queue_duration_s":1.475956,"scheduling_latency_s":1.475973} gitlab # [ 234.965028] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.115Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"feec82a5314e7865d44025c6","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-feec82a5314e7865d44025c6: done: 0.015851 sec","job_status":"done","queue_duration_s":1.475956,"scheduling_latency_s":1.475973,"redis_calls":13,"redis_duration_s":0.002793,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001822,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015851,"completed_at":"2026-09-14T08:38:38.115Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.003790] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.116Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dbd04513ee3132a6ff7b9ae","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dbd04513ee3132a6ff7b9ae: start","job_status":"start","queue_duration_s":1.493244,"scheduling_latency_s":1.493261} gitlab # [ 235.016157] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.132Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8dbd04513ee3132a6ff7b9ae","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8dbd04513ee3132a6ff7b9ae: done: 0.015542 sec","job_status":"done","queue_duration_s":1.493244,"scheduling_latency_s":1.493261,"redis_calls":13,"redis_duration_s":0.0027429999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000945,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001798,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009861,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015542,"completed_at":"2026-09-14T08:38:38.131Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.054438] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b8541e1998d5f973b899c74","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b8541e1998d5f973b899c74: start","job_status":"start","queue_duration_s":1.510144,"scheduling_latency_s":1.510161} gitlab # [ 235.066791] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0b8541e1998d5f973b899c74","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0b8541e1998d5f973b899c74: done: 0.016204 sec","job_status":"done","queue_duration_s":1.510144,"scheduling_latency_s":1.510161,"redis_calls":13,"redis_duration_s":0.003189,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010018,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016204,"completed_at":"2026-09-14T08:38:38.149Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.106846] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.151Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"babd1809c2cd3344ab828ded","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-babd1809c2cd3344ab828ded: start","job_status":"start","queue_duration_s":1.528079,"scheduling_latency_s":1.528096} gitlab # [ 235.119101] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.167Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"babd1809c2cd3344ab828ded","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-babd1809c2cd3344ab828ded: done: 0.01595 sec","job_status":"done","queue_duration_s":1.528079,"scheduling_latency_s":1.528096,"redis_calls":13,"redis_duration_s":0.002974,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009931,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01595,"completed_at":"2026-09-14T08:38:38.167Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.157914] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"709b02fc0576771354bfd9aa","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-709b02fc0576771354bfd9aa: start","job_status":"start","queue_duration_s":1.545548,"scheduling_latency_s":1.545566} gitlab # [ 235.170429] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"709b02fc0576771354bfd9aa","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-709b02fc0576771354bfd9aa: done: 0.01553 sec","job_status":"done","queue_duration_s":1.545548,"scheduling_latency_s":1.545566,"redis_calls":13,"redis_duration_s":0.0026750000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000915,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00176,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010031,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01553,"completed_at":"2026-09-14T08:38:38.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.209274] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4629a4dda250ea762be547f4","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4629a4dda250ea762be547f4: start","job_status":"start","queue_duration_s":1.562687,"scheduling_latency_s":1.562704} gitlab # [ 235.221557] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4629a4dda250ea762be547f4","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4629a4dda250ea762be547f4: done: 0.016032 sec","job_status":"done","queue_duration_s":1.562687,"scheduling_latency_s":1.562704,"redis_calls":13,"redis_duration_s":0.002546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001753,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010741,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016032,"completed_at":"2026-09-14T08:38:38.201Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.259915] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aef2f1a0901d326326741be2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aef2f1a0901d326326741be2: start","job_status":"start","queue_duration_s":1.580333,"scheduling_latency_s":1.580352} gitlab # [ 235.272386] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aef2f1a0901d326326741be2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aef2f1a0901d326326741be2: done: 0.016184 sec","job_status":"done","queue_duration_s":1.580333,"scheduling_latency_s":1.580352,"redis_calls":13,"redis_duration_s":0.00162,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011667,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016184,"completed_at":"2026-09-14T08:38:38.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.312343] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"837ab7c3e81c632d9fc5dafa","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-837ab7c3e81c632d9fc5dafa: start","job_status":"start","queue_duration_s":1.597915,"scheduling_latency_s":1.597934} gitlab # [ 235.324524] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.236Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"837ab7c3e81c632d9fc5dafa","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-837ab7c3e81c632d9fc5dafa: done: 0.015553 sec","job_status":"done","queue_duration_s":1.597915,"scheduling_latency_s":1.597934,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001034,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011151,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015553,"completed_at":"2026-09-14T08:38:38.236Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.362774] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6e038988a3607465400a0cd","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6e038988a3607465400a0cd: start","job_status":"start","queue_duration_s":1.614735,"scheduling_latency_s":1.614754} gitlab # [ 235.375168] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b6e038988a3607465400a0cd","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b6e038988a3607465400a0cd: done: 0.015983 sec","job_status":"done","queue_duration_s":1.614735,"scheduling_latency_s":1.614754,"redis_calls":13,"redis_duration_s":0.002019,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010901,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015983,"completed_at":"2026-09-14T08:38:38.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.414273] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.255Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43777c146545eaf96116f885","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43777c146545eaf96116f885: start","job_status":"start","queue_duration_s":1.631939,"scheduling_latency_s":1.631957} gitlab # [ 235.426665] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43777c146545eaf96116f885","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43777c146545eaf96116f885: done: 0.015382 sec","job_status":"done","queue_duration_s":1.631939,"scheduling_latency_s":1.631957,"redis_calls":13,"redis_duration_s":0.001494,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015382,"completed_at":"2026-09-14T08:38:38.270Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.465369] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.271Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8b8fdd5ed265d2c9b47e142","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8b8fdd5ed265d2c9b47e142: start","job_status":"start","queue_duration_s":1.648409,"scheduling_latency_s":1.648426} gitlab # [ 235.477918] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d8b8fdd5ed265d2c9b47e142","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d8b8fdd5ed265d2c9b47e142: done: 0.015064 sec","job_status":"done","queue_duration_s":1.648409,"scheduling_latency_s":1.648426,"redis_calls":13,"redis_duration_s":0.001673,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001123,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010322,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015064,"completed_at":"2026-09-14T08:38:38.286Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.518115] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.287Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f37f0678dd4c69b33b4ef0b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f37f0678dd4c69b33b4ef0b: start","job_status":"start","queue_duration_s":1.664638,"scheduling_latency_s":1.664657} gitlab # [ 235.530271] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f37f0678dd4c69b33b4ef0b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f37f0678dd4c69b33b4ef0b: done: 0.014928 sec","job_status":"done","queue_duration_s":1.664638,"scheduling_latency_s":1.664657,"redis_calls":13,"redis_duration_s":0.0014759999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010209,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014928,"completed_at":"2026-09-14T08:38:38.302Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.568971] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.303Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cba73f09467d1eea33fea1f7","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cba73f09467d1eea33fea1f7: start","job_status":"start","queue_duration_s":1.680614,"scheduling_latency_s":1.680631} gitlab # [ 235.581305] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.318Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cba73f09467d1eea33fea1f7","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cba73f09467d1eea33fea1f7: done: 0.015022 sec","job_status":"done","queue_duration_s":1.680614,"scheduling_latency_s":1.680631,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000718,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010287,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015022,"completed_at":"2026-09-14T08:38:38.318Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.619971] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.320Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fb1c74cc7d03077b141a0e2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fb1c74cc7d03077b141a0e2: start","job_status":"start","queue_duration_s":1.697577,"scheduling_latency_s":1.697597} gitlab # [ 235.632474] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.335Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7fb1c74cc7d03077b141a0e2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7fb1c74cc7d03077b141a0e2: done: 0.015079 sec","job_status":"done","queue_duration_s":1.697577,"scheduling_latency_s":1.697597,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010565,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015079,"completed_at":"2026-09-14T08:38:38.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.670670] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.337Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd866f05e4cd9adfacf07127","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd866f05e4cd9adfacf07127: start","job_status":"start","queue_duration_s":1.713949,"scheduling_latency_s":1.713966} gitlab # [ 235.682790] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dd866f05e4cd9adfacf07127","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dd866f05e4cd9adfacf07127: done: 0.015193 sec","job_status":"done","queue_duration_s":1.713949,"scheduling_latency_s":1.713966,"redis_calls":13,"redis_duration_s":0.0014290000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010718,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015193,"completed_at":"2026-09-14T08:38:38.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.723282] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd6678dd0eba2760b92c999","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd6678dd0eba2760b92c999: start","job_status":"start","queue_duration_s":1.730406,"scheduling_latency_s":1.730425} gitlab # [ 235.735619] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3dd6678dd0eba2760b92c999","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3dd6678dd0eba2760b92c999: done: 0.014679 sec","job_status":"done","queue_duration_s":1.730406,"scheduling_latency_s":1.730425,"redis_calls":13,"redis_duration_s":0.0014039999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010425,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014679,"completed_at":"2026-09-14T08:38:38.368Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.774305] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.369Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc296efd1a09e269540d8168","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc296efd1a09e269540d8168: start","job_status":"start","queue_duration_s":1.746566,"scheduling_latency_s":1.746583} gitlab # [ 235.786591] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dc296efd1a09e269540d8168","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dc296efd1a09e269540d8168: done: 0.014635 sec","job_status":"done","queue_duration_s":1.746566,"scheduling_latency_s":1.746583,"redis_calls":13,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01039,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014635,"completed_at":"2026-09-14T08:38:38.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.825090] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.385Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e6803b55c35adefe0347c86","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e6803b55c35adefe0347c86: start","job_status":"start","queue_duration_s":1.762491,"scheduling_latency_s":1.76251} gitlab # [ 235.837317] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e6803b55c35adefe0347c86","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e6803b55c35adefe0347c86: done: 0.102248 sec","job_status":"done","queue_duration_s":1.762491,"scheduling_latency_s":1.76251,"redis_calls":13,"redis_duration_s":0.0017039999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001117,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097252,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.102248,"completed_at":"2026-09-14T08:38:38.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.875879] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.489Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe199caa1f9a35a697c115e5","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe199caa1f9a35a697c115e5: start","job_status":"start","queue_duration_s":1.866002,"scheduling_latency_s":1.866022} gitlab # [ 235.887982] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe199caa1f9a35a697c115e5","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe199caa1f9a35a697c115e5: done: 0.015346 sec","job_status":"done","queue_duration_s":1.866002,"scheduling_latency_s":1.866022,"redis_calls":13,"redis_duration_s":0.0015119999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010856,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015346,"completed_at":"2026-09-14T08:38:38.504Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.928043] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442ee729e65aadd1da4b10ee","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442ee729e65aadd1da4b10ee: start","job_status":"start","queue_duration_s":1.882516,"scheduling_latency_s":1.882534} gitlab # [ 235.940149] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"442ee729e65aadd1da4b10ee","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-442ee729e65aadd1da4b10ee: done: 0.014346 sec","job_status":"done","queue_duration_s":1.882516,"scheduling_latency_s":1.882534,"redis_calls":13,"redis_duration_s":0.001464,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014346,"completed_at":"2026-09-14T08:38:38.520Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 235.978308] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04fd6f3ea57aab46e217921b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04fd6f3ea57aab46e217921b: start","job_status":"start","queue_duration_s":1.898552,"scheduling_latency_s":1.89857} gitlab # [ 235.990577] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04fd6f3ea57aab46e217921b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04fd6f3ea57aab46e217921b: done: 0.01494 sec","job_status":"done","queue_duration_s":1.898552,"scheduling_latency_s":1.89857,"redis_calls":13,"redis_duration_s":0.001471,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010303,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01494,"completed_at":"2026-09-14T08:38:38.536Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.028971] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea610c95b80254be22b80aef","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea610c95b80254be22b80aef: start","job_status":"start","queue_duration_s":1.914643,"scheduling_latency_s":1.914661} gitlab # [ 236.040843] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea610c95b80254be22b80aef","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea610c95b80254be22b80aef: done: 0.015077 sec","job_status":"done","queue_duration_s":1.914643,"scheduling_latency_s":1.914661,"redis_calls":13,"redis_duration_s":0.001642,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015077,"completed_at":"2026-09-14T08:38:38.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.079905] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13a946164e908d9ab7a249f2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13a946164e908d9ab7a249f2: start","job_status":"start","queue_duration_s":1.930753,"scheduling_latency_s":1.930769} gitlab # [ 236.092474] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13a946164e908d9ab7a249f2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13a946164e908d9ab7a249f2: done: 0.014605 sec","job_status":"done","queue_duration_s":1.930753,"scheduling_latency_s":1.930769,"redis_calls":13,"redis_duration_s":0.002056,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000924,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010283,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014605,"completed_at":"2026-09-14T08:38:38.568Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.132947] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"745b9fdb88867cbe53158835","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-745b9fdb88867cbe53158835: start","job_status":"start","queue_duration_s":1.946444,"scheduling_latency_s":1.946461} gitlab # [ 236.169758] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"745b9fdb88867cbe53158835","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-745b9fdb88867cbe53158835: done: 0.014571 sec","job_status":"done","queue_duration_s":1.946444,"scheduling_latency_s":1.946461,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010281,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014571,"completed_at":"2026-09-14T08:38:38.584Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.239956] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce8025c5980752f2fc6f043b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce8025c5980752f2fc6f043b: start","job_status":"start","queue_duration_s":1.962105,"scheduling_latency_s":1.962121} gitlab # [ 236.254128] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce8025c5980752f2fc6f043b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce8025c5980752f2fc6f043b: done: 0.014834 sec","job_status":"done","queue_duration_s":1.962105,"scheduling_latency_s":1.962121,"redis_calls":13,"redis_duration_s":0.0013670000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000863,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014834,"completed_at":"2026-09-14T08:38:38.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.293845] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4257c96c1a1bbb8358c89e0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4257c96c1a1bbb8358c89e0: start","job_status":"start","queue_duration_s":1.978028,"scheduling_latency_s":1.978045} gitlab # [ 236.307080] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4257c96c1a1bbb8358c89e0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4257c96c1a1bbb8358c89e0: done: 0.014922 sec","job_status":"done","queue_duration_s":1.978028,"scheduling_latency_s":1.978045,"redis_calls":13,"redis_duration_s":0.001488,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014922,"completed_at":"2026-09-14T08:38:38.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.345656] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44108f8ce922fce152df3b32","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44108f8ce922fce152df3b32: start","job_status":"start","queue_duration_s":1.994083,"scheduling_latency_s":1.994101} gitlab # [ 236.357866] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"44108f8ce922fce152df3b32","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-44108f8ce922fce152df3b32: done: 0.014417 sec","job_status":"done","queue_duration_s":1.994083,"scheduling_latency_s":1.994101,"redis_calls":13,"redis_duration_s":0.001387,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000883,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014417,"completed_at":"2026-09-14T08:38:38.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.396199] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afca8961a6f3bc63c99e2dc2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afca8961a6f3bc63c99e2dc2: start","job_status":"start","queue_duration_s":2.009677,"scheduling_latency_s":2.009693} gitlab # [ 236.408339] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afca8961a6f3bc63c99e2dc2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afca8961a6f3bc63c99e2dc2: done: 0.015083 sec","job_status":"done","queue_duration_s":2.009677,"scheduling_latency_s":2.009693,"redis_calls":13,"redis_duration_s":0.0014169999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015083,"completed_at":"2026-09-14T08:38:38.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.446692] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08488f399e05430845e3b326","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08488f399e05430845e3b326: start","job_status":"start","queue_duration_s":2.025905,"scheduling_latency_s":2.025923} gitlab # [ 236.458885] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.663Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08488f399e05430845e3b326","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08488f399e05430845e3b326: done: 0.01455 sec","job_status":"done","queue_duration_s":2.025905,"scheduling_latency_s":2.025923,"redis_calls":13,"redis_duration_s":0.0013670000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009584,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01455,"completed_at":"2026-09-14T08:38:38.663Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.497862] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76f383da37c3ab65d4175de3","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76f383da37c3ab65d4175de3: start","job_status":"start","queue_duration_s":2.042401,"scheduling_latency_s":2.042418} gitlab # [ 236.511349] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76f383da37c3ab65d4175de3","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76f383da37c3ab65d4175de3: done: 0.014718 sec","job_status":"done","queue_duration_s":2.042401,"scheduling_latency_s":2.042418,"redis_calls":13,"redis_duration_s":0.001395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000896,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014718,"completed_at":"2026-09-14T08:38:38.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.549625] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2de4c7d2fb7f8bcc464c731","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2de4c7d2fb7f8bcc464c731: start","job_status":"start","queue_duration_s":2.058177,"scheduling_latency_s":2.058194} gitlab # [ 236.561632] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2de4c7d2fb7f8bcc464c731","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2de4c7d2fb7f8bcc464c731: done: 0.01513 sec","job_status":"done","queue_duration_s":2.058177,"scheduling_latency_s":2.058194,"redis_calls":13,"redis_duration_s":0.0014659999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000928,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010855,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01513,"completed_at":"2026-09-14T08:38:38.696Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.600259] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c63f8f035d9d825853ae9f4f","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c63f8f035d9d825853ae9f4f: start","job_status":"start","queue_duration_s":2.074435,"scheduling_latency_s":2.074452} gitlab # [ 236.612161] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c63f8f035d9d825853ae9f4f","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c63f8f035d9d825853ae9f4f: done: 0.015277 sec","job_status":"done","queue_duration_s":2.074435,"scheduling_latency_s":2.074452,"redis_calls":13,"redis_duration_s":0.0013930000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01043,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015277,"completed_at":"2026-09-14T08:38:38.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.650319] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9071a48d604eb91cb1586ec1","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9071a48d604eb91cb1586ec1: start","job_status":"start","queue_duration_s":2.090738,"scheduling_latency_s":2.090755} gitlab # [ 236.662414] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9071a48d604eb91cb1586ec1","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9071a48d604eb91cb1586ec1: done: 0.015257 sec","job_status":"done","queue_duration_s":2.090738,"scheduling_latency_s":2.090755,"redis_calls":13,"redis_duration_s":0.001366,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015257,"completed_at":"2026-09-14T08:38:38.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.700603] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73e9db945f7d1bbcc55156f2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73e9db945f7d1bbcc55156f2: start","job_status":"start","queue_duration_s":2.107107,"scheduling_latency_s":2.107124} gitlab # [ 236.713508] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.745Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"73e9db945f7d1bbcc55156f2","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-73e9db945f7d1bbcc55156f2: done: 0.014711 sec","job_status":"done","queue_duration_s":2.107107,"scheduling_latency_s":2.107124,"redis_calls":13,"redis_duration_s":0.001393,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000859,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014711,"completed_at":"2026-09-14T08:38:38.745Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.752150] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21e3f8ac888f817e26c3b843","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21e3f8ac888f817e26c3b843: start","job_status":"start","queue_duration_s":2.122853,"scheduling_latency_s":2.122869} gitlab # [ 236.764146] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.761Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"21e3f8ac888f817e26c3b843","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-21e3f8ac888f817e26c3b843: done: 0.015127 sec","job_status":"done","queue_duration_s":2.122853,"scheduling_latency_s":2.122869,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000917,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010494,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015127,"completed_at":"2026-09-14T08:38:38.761Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.802169] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d3df858586e0f4d87ebe011","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d3df858586e0f4d87ebe011: start","job_status":"start","queue_duration_s":2.139012,"scheduling_latency_s":2.139029} gitlab # [ 236.814351] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.789Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5d3df858586e0f4d87ebe011","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5d3df858586e0f4d87ebe011: done: 0.027495 sec","job_status":"done","queue_duration_s":2.139012,"scheduling_latency_s":2.139029,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027495,"completed_at":"2026-09-14T08:38:38.789Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.852364] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad36e3bfebb57df99e7117ae","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad36e3bfebb57df99e7117ae: start","job_status":"start","queue_duration_s":2.169638,"scheduling_latency_s":2.169682} gitlab # [ 236.864356] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ad36e3bfebb57df99e7117ae","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ad36e3bfebb57df99e7117ae: done: 0.038643 sec","job_status":"done","queue_duration_s":2.169638,"scheduling_latency_s":2.169682,"redis_calls":13,"redis_duration_s":0.0034809999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025802,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.038643,"completed_at":"2026-09-14T08:38:38.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.902583] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce22aefdf93db2dbf129693b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce22aefdf93db2dbf129693b: start","job_status":"start","queue_duration_s":2.212588,"scheduling_latency_s":2.212628} gitlab # [ 236.915579] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.868Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ce22aefdf93db2dbf129693b","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ce22aefdf93db2dbf129693b: done: 0.033125 sec","job_status":"done","queue_duration_s":2.212588,"scheduling_latency_s":2.212628,"redis_calls":13,"redis_duration_s":0.003992,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001407,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002585,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033125,"completed_at":"2026-09-14T08:38:38.868Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 236.953813] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b942e4503155c0208fc2ffd0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b942e4503155c0208fc2ffd0: start","job_status":"start","queue_duration_s":2.248121,"scheduling_latency_s":2.248152} gitlab # [ 236.966015] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.897Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b942e4503155c0208fc2ffd0","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b942e4503155c0208fc2ffd0: done: 0.026597 sec","job_status":"done","queue_duration_s":2.248121,"scheduling_latency_s":2.248152,"redis_calls":13,"redis_duration_s":0.003377,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001155,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002222,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.026597,"completed_at":"2026-09-14T08:38:38.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.004257] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5444812b358192c0aa974b4c","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5444812b358192c0aa974b4c: start","job_status":"start","queue_duration_s":2.276432,"scheduling_latency_s":2.276455} gitlab # [ 237.016173] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.920Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5444812b358192c0aa974b4c","created_at":"2026-09-14T08:38:36.166Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-5e6ac3b26bea49de","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"f427bc2b6718b2611eb22e9bf8edf963","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.623Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5444812b358192c0aa974b4c: done: 0.020812 sec","job_status":"done","queue_duration_s":2.276432,"scheduling_latency_s":2.276455,"redis_calls":13,"redis_duration_s":0.002929,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001013,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020812,"completed_at":"2026-09-14T08:38:38.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.054387] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.922Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"33314c22510e6576d72150a1","created_at":"2026-09-14T08:38:36.626Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95511158d23a22834cf61343bd5d6abe","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.656Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-33314c22510e6576d72150a1: start","job_status":"start","queue_duration_s":2.265882,"scheduling_latency_s":2.265902} gitlab # [ 237.068629] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.957Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"33314c22510e6576d72150a1","created_at":"2026-09-14T08:38:36.626Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"95511158d23a22834cf61343bd5d6abe","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.656Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-33314c22510e6576d72150a1: done: 0.034944 sec","job_status":"done","queue_duration_s":2.265882,"scheduling_latency_s":2.265902,"redis_calls":10,"redis_duration_s":0.002606,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000504,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000542,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021198,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00037658,"duration_s":0.034944,"completed_at":"2026-09-14T08:38:38.957Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003817,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.109924] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3891dc70d9021e7356a7265","created_at":"2026-09-14T08:38:36.739Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"894c4fd6113398b52d682e181a4eebcf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.767Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3891dc70d9021e7356a7265: start","job_status":"start","queue_duration_s":2.191469,"scheduling_latency_s":2.191488} gitlab # [ 237.124910] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.979Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"e3891dc70d9021e7356a7265","created_at":"2026-09-14T08:38:36.739Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"894c4fd6113398b52d682e181a4eebcf","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.767Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-e3891dc70d9021e7356a7265: done: 0.020852 sec","job_status":"done","queue_duration_s":2.191469,"scheduling_latency_s":2.191488,"redis_calls":10,"redis_duration_s":0.00241,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001427,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000511,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012934,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036306,"duration_s":0.020852,"completed_at":"2026-09-14T08:38:38.979Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00047,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.166166] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:38.981Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"954a00872b7a18c48548b5e6","created_at":"2026-09-14T08:38:36.837Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a30f523e4f05c2e5b7e9df9872229223","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.861Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-954a00872b7a18c48548b5e6: start","job_status":"start","queue_duration_s":2.119425,"scheduling_latency_s":2.119441} gitlab # [ 237.180288] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.049Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"954a00872b7a18c48548b5e6","created_at":"2026-09-14T08:38:36.837Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a30f523e4f05c2e5b7e9df9872229223","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.861Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-954a00872b7a18c48548b5e6: done: 0.068673 sec","job_status":"done","queue_duration_s":2.119425,"scheduling_latency_s":2.119441,"redis_calls":14,"redis_duration_s":0.003732,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000896,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001564,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001272,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.025,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00062028,"exclusive_lock_hold_duration_s":0.04454582,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.068673,"completed_at":"2026-09-14T08:38:39.049Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.03042,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.224229] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.051Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"88658a785bc5cd3e94369d77","created_at":"2026-09-14T08:38:36.889Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"128d72ca657103d9c3930d8a8a858dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.916Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-88658a785bc5cd3e94369d77: start","job_status":"start","queue_duration_s":2.135058,"scheduling_latency_s":2.135079} gitlab # [ 237.237213] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.084Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"88658a785bc5cd3e94369d77","created_at":"2026-09-14T08:38:36.889Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"128d72ca657103d9c3930d8a8a858dbd","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.916Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-88658a785bc5cd3e94369d77: done: 0.033201 sec","job_status":"done","queue_duration_s":2.135058,"scheduling_latency_s":2.135079,"redis_calls":12,"redis_duration_s":0.003022,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000493,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001094,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00063804,"exclusive_lock_hold_duration_s":0.00804,"duration_s":0.033201,"completed_at":"2026-09-14T08:38:39.084Z","load_balancing_strategy":"primary","db_duration_s":0.00327,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.277831] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e61752a4506db0476e65501d","created_at":"2026-09-14T08:38:36.923Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eff109c82e9a089dabd6cd216d232784","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.942Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e61752a4506db0476e65501d: start","job_status":"start","queue_duration_s":2.144089,"scheduling_latency_s":2.144105} gitlab # [ 237.292193] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.099Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 237.295827] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.109Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"e61752a4506db0476e65501d","created_at":"2026-09-14T08:38:36.923Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eff109c82e9a089dabd6cd216d232784","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.942Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-e61752a4506db0476e65501d: done: 0.022828 sec","job_status":"done","queue_duration_s":2.144089,"scheduling_latency_s":2.144105,"redis_calls":10,"redis_duration_s":0.004161,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002242,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001435,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000484,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011006,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032412,"duration_s":0.022828,"completed_at":"2026-09-14T08:38:39.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.338385] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9632069e56b5f53e951c9788","created_at":"2026-09-14T08:38:36.953Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eaa2039ac3a8cc2e1e3d33574f9c42ae","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.965Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9632069e56b5f53e951c9788: start","job_status":"start","queue_duration_s":2.145662,"scheduling_latency_s":2.14568} gitlab # [ 237.352228] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.433Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"9632069e56b5f53e951c9788","created_at":"2026-09-14T08:38:36.953Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"eaa2039ac3a8cc2e1e3d33574f9c42ae","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.965Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-9632069e56b5f53e951c9788: done: 0.322761 sec","job_status":"done","queue_duration_s":2.145662,"scheduling_latency_s":2.14568,"redis_calls":565,"redis_duration_s":0.062004,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000472,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.060417,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.293649,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025016,"duration_s":0.322761,"completed_at":"2026-09-14T08:38:39.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000438,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.393367] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"00ad63619ce4425560e529c7","created_at":"2026-09-14T08:38:36.980Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7429dd22a2125a8652413271b5f52da8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.991Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-00ad63619ce4425560e529c7: start","job_status":"start","queue_duration_s":2.4431,"scheduling_latency_s":2.443122} gitlab # [ 237.407546] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.459Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"00ad63619ce4425560e529c7","created_at":"2026-09-14T08:38:36.980Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7429dd22a2125a8652413271b5f52da8","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:36.991Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-00ad63619ce4425560e529c7: done: 0.024957 sec","job_status":"done","queue_duration_s":2.4431,"scheduling_latency_s":2.443122,"redis_calls":12,"redis_duration_s":0.001666,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000284,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000733,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000649,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00048334,"exclusive_lock_hold_duration_s":0.005503,"duration_s":0.024957,"completed_at":"2026-09-14T08:38:39.459Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002386,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.449031] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c5444f21b9e60f52788b0671","created_at":"2026-09-14T08:38:37.010Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a7095884de2a14f1e9c721547fccb0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.018Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c5444f21b9e60f52788b0671: start","job_status":"start","queue_duration_s":2.442239,"scheduling_latency_s":2.442258} gitlab # [ 237.463238] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.472Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 237.466651] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.479Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"c5444f21b9e60f52788b0671","created_at":"2026-09-14T08:38:37.010Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0a7095884de2a14f1e9c721547fccb0f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.018Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-c5444f21b9e60f52788b0671: done: 0.017996 sec","job_status":"done","queue_duration_s":2.442239,"scheduling_latency_s":2.442258,"redis_calls":10,"redis_duration_s":0.001357,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000307,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00078,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011413,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002474,"duration_s":0.017996,"completed_at":"2026-09-14T08:38:39.478Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000326,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.507987] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.480Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ef2d770a09d3e7e9b91e6863","created_at":"2026-09-14T08:38:37.025Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1c64d095d0f4667a624ed24ef049958","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.034Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ef2d770a09d3e7e9b91e6863: start","job_status":"start","queue_duration_s":2.446327,"scheduling_latency_s":2.446346} gitlab # [ 237.522121] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"ef2d770a09d3e7e9b91e6863","created_at":"2026-09-14T08:38:37.025Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1c64d095d0f4667a624ed24ef049958","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.034Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-ef2d770a09d3e7e9b91e6863: done: 0.029865 sec","job_status":"done","queue_duration_s":2.446327,"scheduling_latency_s":2.446346,"redis_calls":13,"redis_duration_s":0.001781,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000274,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.000881,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000626,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01004544,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.029865,"completed_at":"2026-09-14T08:38:39.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002076,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 237.566953] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ce3ceb63c03b6e80b396ed28","created_at":"2026-09-14T08:38:37.084Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23a728c33163f9670341971ab37b1124","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.102Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ce3ceb63c03b6e80b396ed28: start","job_status":"start","queue_duration_s":2.408612,"scheduling_latency_s":2.408631} gitlab # [ 237.582008] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"ce3ceb63c03b6e80b396ed28","created_at":"2026-09-14T08:38:37.084Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"23a728c33163f9670341971ab37b1124","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.102Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-ce3ceb63c03b6e80b396ed28: done: 0.037827 sec","job_status":"done","queue_duration_s":2.408612,"scheduling_latency_s":2.408631,"redis_calls":12,"redis_duration_s":0.001696,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000281,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000732,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000683,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057676,"exclusive_lock_hold_duration_s":0.01256976,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.037827,"completed_at":"2026-09-14T08:38:39.549Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.627031] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.550Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"460375275f48aac75bd2c3c1","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-460375275f48aac75bd2c3c1: start","job_status":"start","queue_duration_s":1.701417,"scheduling_latency_s":1.701434} gitlab # [ 237.639522] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.567Z","message":"No existing merge request to be cleaned up."} gitlab # [ 237.641278] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.581Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"460375275f48aac75bd2c3c1","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-460375275f48aac75bd2c3c1: done: 0.030798 sec","job_status":"done","queue_duration_s":1.701417,"scheduling_latency_s":1.701434,"redis_calls":13,"redis_duration_s":0.00181,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000857,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026648,"duration_s":0.030798,"completed_at":"2026-09-14T08:38:39.581Z","load_balancing_strategy":"primary","db_duration_s":0.002751,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.681029] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.582Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d5ee427715c4eeb23492ce16","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d5ee427715c4eeb23492ce16: start","job_status":"start","queue_duration_s":1.733705,"scheduling_latency_s":1.733724} gitlab # [ 237.693074] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.593Z","message":"No existing merge request to be cleaned up."} gitlab # [ 237.694762] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.602Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d5ee427715c4eeb23492ce16","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d5ee427715c4eeb23492ce16: done: 0.019465 sec","job_status":"done","queue_duration_s":1.733705,"scheduling_latency_s":1.733724,"redis_calls":10,"redis_duration_s":0.0013839999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0006,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019465,"completed_at":"2026-09-14T08:38:39.602Z","load_balancing_strategy":"primary","db_duration_s":0.001307,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.733568] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.603Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"982aeecb7ed973a56e7c66f3","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-982aeecb7ed973a56e7c66f3: start","job_status":"start","queue_duration_s":1.754743,"scheduling_latency_s":1.754762} gitlab # [ 237.745517] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.614Z","message":"No existing merge request to be cleaned up."} gitlab # [ 237.747136] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.623Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"982aeecb7ed973a56e7c66f3","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-982aeecb7ed973a56e7c66f3: done: 0.019659 sec","job_status":"done","queue_duration_s":1.754743,"scheduling_latency_s":1.754762,"redis_calls":10,"redis_duration_s":0.001371,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000798,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019659,"completed_at":"2026-09-14T08:38:39.623Z","load_balancing_strategy":"primary","db_duration_s":0.001288,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 237.785280] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.624Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b69b610997fb7738fcdae653","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b69b610997fb7738fcdae653: start","job_status":"start","queue_duration_s":1.775512,"scheduling_latency_s":1.77553} gitlab # [ 237.797308] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.635Z","message":"No existing merge request to be cleaned up."} gitlab # [ 237.798901] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:38:39.644Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b69b610997fb7738fcdae653","created_at":"2026-09-14T08:38:37.835Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-ac6dff57b5284d36","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"840362ad68b0da1224a6629925fff7fa","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:38:37.849Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b69b610997fb7738fcdae653: done: 0.019767 sec","job_status":"done","queue_duration_s":1.775512,"scheduling_latency_s":1.77553,"redis_calls":10,"redis_duration_s":0.001912,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000824,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019767,"completed_at":"2026-09-14T08:38:39.644Z","load_balancing_strategy":"primary","db_duration_s":0.001279,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.138899] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:06.965Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6dddfe470e3e6ef847f997ff","created_at":"2026-09-14T08:39:06.944Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:06.962Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6dddfe470e3e6ef847f997ff: start","job_status":"start","queue_duration_s":0.003268,"scheduling_latency_s":0.003329} gitlab # [ 263.916173] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:07.743Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"6dddfe470e3e6ef847f997ff","created_at":"2026-09-14T08:39:06.944Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:06.962Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-6dddfe470e3e6ef847f997ff: done: 0.778598 sec","job_status":"done","queue_duration_s":0.003268,"scheduling_latency_s":0.003329,"redis_calls":60,"redis_duration_s":0.12149700000000001,"redis_read_bytes":108,"redis_write_bytes":41059,"redis_queues_calls":2,"redis_queues_duration_s":0.00372,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.113749,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004028,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.256599,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.778598,"completed_at":"2026-09-14T08:39:07.743Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 263.971225] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:07.758Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b85de7b9877c00629d9710dc","created_at":"2026-09-14T08:39:06.974Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd89578e05b259db204badf4a831e829","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:39:06.980Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b85de7b9877c00629d9710dc: start","job_status":"start","queue_duration_s":0.777151,"scheduling_latency_s":0.777207} gitlab # [ 264.023098] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:07.851Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 264.098183] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:07.924Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"b85de7b9877c00629d9710dc","created_at":"2026-09-14T08:39:06.974Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dd89578e05b259db204badf4a831e829","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:39:06.980Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-b85de7b9877c00629d9710dc: done: 0.165874 sec","job_status":"done","queue_duration_s":0.777151,"scheduling_latency_s":0.777207,"redis_calls":7,"redis_duration_s":0.068765,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.065976,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002789,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.165874,"completed_at":"2026-09-14T08:39:07.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.161385] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:07.929Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da1b129561b577664b02a2de","created_at":"2026-09-14T08:39:06.995Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b63aed7af64c9cf3c7e66cbee742f495","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.001Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da1b129561b577664b02a2de: start","job_status":"start","queue_duration_s":0.928141,"scheduling_latency_s":0.928193} gitlab # [ 264.606666] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"da1b129561b577664b02a2de","created_at":"2026-09-14T08:39:06.995Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b63aed7af64c9cf3c7e66cbee742f495","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.001Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-da1b129561b577664b02a2de: done: 0.504653 sec","job_status":"done","queue_duration_s":0.928141,"scheduling_latency_s":0.928193,"redis_calls":9,"redis_duration_s":0.004178,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002145,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.075,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.075,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.401439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00187022,"exclusive_lock_hold_duration_s":0.4697391,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.504653,"completed_at":"2026-09-14T08:39:08.434Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.136755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.702914] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33594c72b63059a721c85b3d","created_at":"2026-09-14T08:39:07.019Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d65bd90ba3762311270c25ccd1ae9811","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.026Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33594c72b63059a721c85b3d: start","job_status":"start","queue_duration_s":1.410537,"scheduling_latency_s":1.410569} gitlab # [ 264.726513] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.457Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"33594c72b63059a721c85b3d","created_at":"2026-09-14T08:39:07.019Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d65bd90ba3762311270c25ccd1ae9811","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.026Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-33594c72b63059a721c85b3d: done: 0.021183 sec","job_status":"done","queue_duration_s":1.410537,"scheduling_latency_s":1.410569,"redis_calls":7,"redis_duration_s":0.002074,"redis_read_bytes":5,"redis_write_bytes":831,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001499,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000575,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021183,"completed_at":"2026-09-14T08:39:08.457Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.773653] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f63c1bd18e61d10257522c28","created_at":"2026-09-14T08:39:07.043Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fa962ed0ddf06513d726f3261971157","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.053Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f63c1bd18e61d10257522c28: start","job_status":"start","queue_duration_s":1.406532,"scheduling_latency_s":1.406557} gitlab # [ 264.787870] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.469Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 264.791562] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.475Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f63c1bd18e61d10257522c28","created_at":"2026-09-14T08:39:07.043Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0fa962ed0ddf06513d726f3261971157","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.053Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f63c1bd18e61d10257522c28: done: 0.015239 sec","job_status":"done","queue_duration_s":1.406532,"scheduling_latency_s":1.406557,"redis_calls":7,"redis_duration_s":0.001697,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000472,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015239,"completed_at":"2026-09-14T08:39:08.475Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.831262] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.477Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ae8f1c64377babcb74c006a0","created_at":"2026-09-14T08:39:07.100Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90bf5183620e4a6ab94ce9071aaee40a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.111Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ae8f1c64377babcb74c006a0: start","job_status":"start","queue_duration_s":1.366046,"scheduling_latency_s":1.366063} gitlab # [ 264.845359] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.518Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"ae8f1c64377babcb74c006a0","created_at":"2026-09-14T08:39:07.100Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90bf5183620e4a6ab94ce9071aaee40a","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.111Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-ae8f1c64377babcb74c006a0: done: 0.040923 sec","job_status":"done","queue_duration_s":1.366046,"scheduling_latency_s":1.366063,"redis_calls":7,"redis_duration_s":0.002317,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001682,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000635,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.040923,"completed_at":"2026-09-14T08:39:08.518Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009795,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.886067] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"04a24e9cef3de7db689c9e43","created_at":"2026-09-14T08:39:07.165Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426d35f3b68894a7ac6559acf0c7a165","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:39:07.176Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-04a24e9cef3de7db689c9e43: start","job_status":"start","queue_duration_s":1.345517,"scheduling_latency_s":1.345561} gitlab # [ 264.899232] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.571Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"04a24e9cef3de7db689c9e43","created_at":"2026-09-14T08:39:07.165Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"426d35f3b68894a7ac6559acf0c7a165","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:39:07.176Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-04a24e9cef3de7db689c9e43: done: 0.04974 sec","job_status":"done","queue_duration_s":1.345517,"scheduling_latency_s":1.345561,"redis_calls":7,"redis_duration_s":0.002814,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001997,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000817,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.035447,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04974,"completed_at":"2026-09-14T08:39:08.571Z","load_balancing_strategy":"primary","db_duration_s":0.003589,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.941923] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.575Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a34b0a75bd654f28289135da","created_at":"2026-09-14T08:39:07.199Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a94006ff0991a0a8270b34c565685dd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.209Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a34b0a75bd654f28289135da: start","job_status":"start","queue_duration_s":1.366128,"scheduling_latency_s":1.366184} gitlab # [ 264.955691] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.627Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a34b0a75bd654f28289135da","created_at":"2026-09-14T08:39:07.199Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1a94006ff0991a0a8270b34c565685dd","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.209Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a34b0a75bd654f28289135da: done: 0.051343 sec","job_status":"done","queue_duration_s":1.366128,"scheduling_latency_s":1.366184,"redis_calls":7,"redis_duration_s":0.004338,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00348,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000858,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022951,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.051343,"completed_at":"2026-09-14T08:39:08.627Z","load_balancing_strategy":"primary","db_duration_s":0.017837,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 264.995598] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.630Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"244607d2f2142785eff218ac","created_at":"2026-09-14T08:39:07.240Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a51161551a6350d242f7fdecb8d30285","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:39:07.251Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-244607d2f2142785eff218ac: start","job_status":"start","queue_duration_s":1.378859,"scheduling_latency_s":1.378898} gitlab # [ 265.008889] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.648Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 265.012529] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"244607d2f2142785eff218ac","created_at":"2026-09-14T08:39:07.240Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a51161551a6350d242f7fdecb8d30285","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:39:07.251Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-244607d2f2142785eff218ac: done: 0.027916 sec","job_status":"done","queue_duration_s":1.378859,"scheduling_latency_s":1.378898,"redis_calls":7,"redis_duration_s":0.002395,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001708,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000687,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027916,"completed_at":"2026-09-14T08:39:08.657Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.051376] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.662Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b20d15bf74ea0e0f68e2866c","created_at":"2026-09-14T08:39:07.275Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2677e4d158fc85b739fe596ecef2b98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.285Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b20d15bf74ea0e0f68e2866c: start","job_status":"start","queue_duration_s":1.376113,"scheduling_latency_s":1.376162} gitlab # [ 265.065781] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.684Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 265.068904] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.697Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b20d15bf74ea0e0f68e2866c","created_at":"2026-09-14T08:39:07.275Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f2677e4d158fc85b739fe596ecef2b98","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.285Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b20d15bf74ea0e0f68e2866c: done: 0.03519 sec","job_status":"done","queue_duration_s":1.376113,"scheduling_latency_s":1.376162,"redis_calls":7,"redis_duration_s":0.003005,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00209,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03519,"completed_at":"2026-09-14T08:39:08.697Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.109625] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.700Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1500ea625d4682c0e389e48f","created_at":"2026-09-14T08:39:07.311Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.320Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1500ea625d4682c0e389e48f: start","job_status":"start","queue_duration_s":1.379611,"scheduling_latency_s":1.379657} gitlab # [ 265.122912] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.769Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1500ea625d4682c0e389e48f","created_at":"2026-09-14T08:39:07.311Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.320Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1500ea625d4682c0e389e48f: done: 0.068614 sec","job_status":"done","queue_duration_s":1.379611,"scheduling_latency_s":1.379657,"redis_calls":14,"redis_duration_s":0.005994,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000461,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004269,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.052498,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.068614,"completed_at":"2026-09-14T08:39:08.769Z","load_balancing_strategy":"primary","db_duration_s":0.002596,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.163246] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.771Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"63ac3db7cae948f5c2960355","created_at":"2026-09-14T08:39:07.341Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85755d42a6de60e3c46d44552135c572","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.350Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-63ac3db7cae948f5c2960355: start","job_status":"start","queue_duration_s":1.420625,"scheduling_latency_s":1.420667} gitlab # [ 265.177275] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"63ac3db7cae948f5c2960355","created_at":"2026-09-14T08:39:07.341Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85755d42a6de60e3c46d44552135c572","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.350Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-63ac3db7cae948f5c2960355: done: 0.02667 sec","job_status":"done","queue_duration_s":1.420625,"scheduling_latency_s":1.420667,"redis_calls":7,"redis_duration_s":0.00163,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00121,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015609,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02667,"completed_at":"2026-09-14T08:39:08.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001759,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.217061] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6b1941f5f54c3debe5953f28","created_at":"2026-09-14T08:39:07.371Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3c69840b56f7db76b9ee2f6725bdb96","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.384Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6b1941f5f54c3debe5953f28: start","job_status":"start","queue_duration_s":1.419569,"scheduling_latency_s":1.419608} gitlab # [ 265.231051] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.833Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"6b1941f5f54c3debe5953f28","created_at":"2026-09-14T08:39:07.371Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e3c69840b56f7db76b9ee2f6725bdb96","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.384Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-6b1941f5f54c3debe5953f28: done: 0.028973 sec","job_status":"done","queue_duration_s":1.419569,"scheduling_latency_s":1.419608,"redis_calls":9,"redis_duration_s":0.0022459999999999997,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00109,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00077392,"exclusive_lock_hold_duration_s":0.00502968,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.028973,"completed_at":"2026-09-14T08:39:08.833Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.271978] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.835Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6ce0c9ebd38095a4af1a8ee9","created_at":"2026-09-14T08:39:07.418Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcf7ce250b204170dba033bad0547a4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.429Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6ce0c9ebd38095a4af1a8ee9: start","job_status":"start","queue_duration_s":1.405985,"scheduling_latency_s":1.406024} gitlab # [ 265.287447] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.856Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"6ce0c9ebd38095a4af1a8ee9","created_at":"2026-09-14T08:39:07.418Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bcf7ce250b204170dba033bad0547a4e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.429Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-6ce0c9ebd38095a4af1a8ee9: done: 0.021433 sec","job_status":"done","queue_duration_s":1.405985,"scheduling_latency_s":1.406024,"redis_calls":7,"redis_duration_s":0.002358,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001963,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000395,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013777,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021433,"completed_at":"2026-09-14T08:39:08.856Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.328402] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.858Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b26681adecf0ac6676895f52","created_at":"2026-09-14T08:39:07.485Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b329462906e0da2bc515eeabfce592f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.495Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b26681adecf0ac6676895f52: start","job_status":"start","queue_duration_s":1.362729,"scheduling_latency_s":1.362751} gitlab # [ 265.342471] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.912Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"b26681adecf0ac6676895f52","created_at":"2026-09-14T08:39:07.485Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b329462906e0da2bc515eeabfce592f8","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.495Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-b26681adecf0ac6676895f52: done: 0.05378 sec","job_status":"done","queue_duration_s":1.362729,"scheduling_latency_s":1.362751,"redis_calls":9,"redis_duration_s":0.001856,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000855,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.017,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005849,"exclusive_lock_hold_duration_s":0.03268864,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.05378,"completed_at":"2026-09-14T08:39:08.912Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.018793,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.384185] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.914Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0e1b698a9c040335a4ba5e74","created_at":"2026-09-14T08:39:07.531Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00dddfb185657340878c5978a41a325f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.540Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0e1b698a9c040335a4ba5e74: start","job_status":"start","queue_duration_s":1.373469,"scheduling_latency_s":1.373495} gitlab # [ 265.397314] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.940Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"0e1b698a9c040335a4ba5e74","created_at":"2026-09-14T08:39:07.531Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"00dddfb185657340878c5978a41a325f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.540Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-0e1b698a9c040335a4ba5e74: done: 0.026481 sec","job_status":"done","queue_duration_s":1.373469,"scheduling_latency_s":1.373495,"redis_calls":9,"redis_duration_s":0.0018249999999999998,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000819,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00069454,"exclusive_lock_hold_duration_s":0.00526752,"duration_s":0.026481,"completed_at":"2026-09-14T08:39:08.940Z","load_balancing_strategy":"primary","db_duration_s":0.00212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.437466] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.942Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f108d106d145866e6466f50c","created_at":"2026-09-14T08:39:07.560Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5775302cb670a3daa5c0147df36d391b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.570Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f108d106d145866e6466f50c: start","job_status":"start","queue_duration_s":1.371421,"scheduling_latency_s":1.371443} gitlab # [ 265.451833] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.951Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 265.455796] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.958Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"f108d106d145866e6466f50c","created_at":"2026-09-14T08:39:07.560Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5775302cb670a3daa5c0147df36d391b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.570Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-f108d106d145866e6466f50c: done: 0.016505 sec","job_status":"done","queue_duration_s":1.371421,"scheduling_latency_s":1.371443,"redis_calls":7,"redis_duration_s":0.001199,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000869,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00033,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.016505,"completed_at":"2026-09-14T08:39:08.958Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.497160] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d079debcce67ffd63ff93f3b","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d079debcce67ffd63ff93f3b: start","job_status":"start","queue_duration_s":1.365054,"scheduling_latency_s":1.365074} gitlab # [ 265.509287] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d079debcce67ffd63ff93f3b","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d079debcce67ffd63ff93f3b: done: 0.018518 sec","job_status":"done","queue_duration_s":1.365054,"scheduling_latency_s":1.365074,"redis_calls":13,"redis_duration_s":0.001686,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001085,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011853,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018518,"completed_at":"2026-09-14T08:39:08.978Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.547782] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.980Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fe88aa7857557ac57560bdb","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fe88aa7857557ac57560bdb: start","job_status":"start","queue_duration_s":1.3849,"scheduling_latency_s":1.384918} gitlab # [ 265.560195] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.995Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fe88aa7857557ac57560bdb","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fe88aa7857557ac57560bdb: done: 0.015861 sec","job_status":"done","queue_duration_s":1.3849,"scheduling_latency_s":1.384918,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015861,"completed_at":"2026-09-14T08:39:08.995Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.598481] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:08.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b77b1a39c0dff806d37eee3a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b77b1a39c0dff806d37eee3a: start","job_status":"start","queue_duration_s":1.401997,"scheduling_latency_s":1.402013} gitlab # [ 265.610813] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.012Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b77b1a39c0dff806d37eee3a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b77b1a39c0dff806d37eee3a: done: 0.01529 sec","job_status":"done","queue_duration_s":1.401997,"scheduling_latency_s":1.402013,"redis_calls":13,"redis_duration_s":0.001457,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01529,"completed_at":"2026-09-14T08:39:09.012Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.649105] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4178296810dd44d700bccb4","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4178296810dd44d700bccb4: start","job_status":"start","queue_duration_s":1.418574,"scheduling_latency_s":1.418592} gitlab # [ 265.661434] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4178296810dd44d700bccb4","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4178296810dd44d700bccb4: done: 0.015769 sec","job_status":"done","queue_duration_s":1.418574,"scheduling_latency_s":1.418592,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015769,"completed_at":"2026-09-14T08:39:09.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.701010] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.030Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d20d63b05516cb9652204412","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d20d63b05516cb9652204412: start","job_status":"start","queue_duration_s":1.435537,"scheduling_latency_s":1.435554} gitlab # [ 265.713312] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d20d63b05516cb9652204412","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d20d63b05516cb9652204412: done: 0.014573 sec","job_status":"done","queue_duration_s":1.435537,"scheduling_latency_s":1.435554,"redis_calls":13,"redis_duration_s":0.001407,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010088,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014573,"completed_at":"2026-09-14T08:39:09.045Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.751690] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51303b8bf26d5e59419f4904","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51303b8bf26d5e59419f4904: start","job_status":"start","queue_duration_s":1.451324,"scheduling_latency_s":1.451341} gitlab # [ 265.764077] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"51303b8bf26d5e59419f4904","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-51303b8bf26d5e59419f4904: done: 0.014545 sec","job_status":"done","queue_duration_s":1.451324,"scheduling_latency_s":1.451341,"redis_calls":13,"redis_duration_s":0.001418,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010055,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014545,"completed_at":"2026-09-14T08:39:09.060Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.802678] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.062Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7ecb3912249deaf76138c22","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7ecb3912249deaf76138c22: start","job_status":"start","queue_duration_s":1.467335,"scheduling_latency_s":1.467352} gitlab # [ 265.814810] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.077Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e7ecb3912249deaf76138c22","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e7ecb3912249deaf76138c22: done: 0.0151 sec","job_status":"done","queue_duration_s":1.467335,"scheduling_latency_s":1.467352,"redis_calls":13,"redis_duration_s":0.001385,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000511,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000874,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0151,"completed_at":"2026-09-14T08:39:09.077Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.853664] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14b26f0356bc3b3af62013aa","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14b26f0356bc3b3af62013aa: start","job_status":"start","queue_duration_s":1.483625,"scheduling_latency_s":1.483641} gitlab # [ 265.866067] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14b26f0356bc3b3af62013aa","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14b26f0356bc3b3af62013aa: done: 0.014749 sec","job_status":"done","queue_duration_s":1.483625,"scheduling_latency_s":1.483641,"redis_calls":13,"redis_duration_s":0.001836,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001329,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010187,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014749,"completed_at":"2026-09-14T08:39:09.093Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.905708] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.094Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"612575a2ae0147b47ece7de1","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-612575a2ae0147b47ece7de1: start","job_status":"start","queue_duration_s":1.499548,"scheduling_latency_s":1.499565} gitlab # [ 265.917980] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.109Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"612575a2ae0147b47ece7de1","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-612575a2ae0147b47ece7de1: done: 0.014621 sec","job_status":"done","queue_duration_s":1.499548,"scheduling_latency_s":1.499565,"redis_calls":13,"redis_duration_s":0.00138,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000877,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010141,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014621,"completed_at":"2026-09-14T08:39:09.109Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 265.956510] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604afc93968222b7875212c9","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604afc93968222b7875212c9: start","job_status":"start","queue_duration_s":1.515381,"scheduling_latency_s":1.515398} gitlab # [ 265.968673] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"604afc93968222b7875212c9","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-604afc93968222b7875212c9: done: 0.015217 sec","job_status":"done","queue_duration_s":1.515381,"scheduling_latency_s":1.515398,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00091,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015217,"completed_at":"2026-09-14T08:39:09.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.007276] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13cfba616e99502a59ad8e9a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13cfba616e99502a59ad8e9a: start","job_status":"start","queue_duration_s":1.531791,"scheduling_latency_s":1.531808} gitlab # [ 266.019763] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"13cfba616e99502a59ad8e9a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-13cfba616e99502a59ad8e9a: done: 0.01414 sec","job_status":"done","queue_duration_s":1.531791,"scheduling_latency_s":1.531808,"redis_calls":13,"redis_duration_s":0.00139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000882,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00984,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01414,"completed_at":"2026-09-14T08:39:09.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.058583] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a33c4e914300e6307b380fc","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a33c4e914300e6307b380fc: start","job_status":"start","queue_duration_s":1.546974,"scheduling_latency_s":1.546991} gitlab # [ 266.070858] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a33c4e914300e6307b380fc","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a33c4e914300e6307b380fc: done: 0.014224 sec","job_status":"done","queue_duration_s":1.546974,"scheduling_latency_s":1.546991,"redis_calls":13,"redis_duration_s":0.001385,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000871,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014224,"completed_at":"2026-09-14T08:39:09.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.109981] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68dfa13d78525ac98eef1460","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68dfa13d78525ac98eef1460: start","job_status":"start","queue_duration_s":1.562345,"scheduling_latency_s":1.562362} gitlab # [ 266.121968] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"68dfa13d78525ac98eef1460","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-68dfa13d78525ac98eef1460: done: 0.014466 sec","job_status":"done","queue_duration_s":1.562345,"scheduling_latency_s":1.562362,"redis_calls":13,"redis_duration_s":0.001667,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001135,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014466,"completed_at":"2026-09-14T08:39:09.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.159971] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bee106f12cc391a0204a8b7d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bee106f12cc391a0204a8b7d: start","job_status":"start","queue_duration_s":1.577912,"scheduling_latency_s":1.577929} gitlab # [ 266.172427] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.189Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bee106f12cc391a0204a8b7d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bee106f12cc391a0204a8b7d: done: 0.016627 sec","job_status":"done","queue_duration_s":1.577912,"scheduling_latency_s":1.577929,"redis_calls":13,"redis_duration_s":0.001472,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012393,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016627,"completed_at":"2026-09-14T08:39:09.189Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.210164] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.190Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c06e9ae3bfe7133fa50726","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c06e9ae3bfe7133fa50726: start","job_status":"start","queue_duration_s":1.595655,"scheduling_latency_s":1.595672} gitlab # [ 266.222371] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.205Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"02c06e9ae3bfe7133fa50726","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-02c06e9ae3bfe7133fa50726: done: 0.014485 sec","job_status":"done","queue_duration_s":1.595655,"scheduling_latency_s":1.595672,"redis_calls":13,"redis_duration_s":0.0014160000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014485,"completed_at":"2026-09-14T08:39:09.205Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.260269] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.206Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee659d7f23a11e289df472df","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee659d7f23a11e289df472df: start","job_status":"start","queue_duration_s":1.611366,"scheduling_latency_s":1.611383} gitlab # [ 266.272390] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee659d7f23a11e289df472df","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee659d7f23a11e289df472df: done: 0.014562 sec","job_status":"done","queue_duration_s":1.611366,"scheduling_latency_s":1.611383,"redis_calls":13,"redis_duration_s":0.001454,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010223,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-09-14T08:39:09.221Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.311089] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.222Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca64915cc5c5a99d9892d87d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca64915cc5c5a99d9892d87d: start","job_status":"start","queue_duration_s":1.627011,"scheduling_latency_s":1.627028} gitlab # [ 266.323292] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ca64915cc5c5a99d9892d87d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ca64915cc5c5a99d9892d87d: done: 0.015147 sec","job_status":"done","queue_duration_s":1.627011,"scheduling_latency_s":1.627028,"redis_calls":13,"redis_duration_s":0.001435,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010308,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015147,"completed_at":"2026-09-14T08:39:09.237Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.361217] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5cc73a2f8feff6a8264a42","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5cc73a2f8feff6a8264a42: start","job_status":"start","queue_duration_s":1.643295,"scheduling_latency_s":1.643312} gitlab # [ 266.373233] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.253Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e5cc73a2f8feff6a8264a42","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e5cc73a2f8feff6a8264a42: done: 0.014678 sec","job_status":"done","queue_duration_s":1.643295,"scheduling_latency_s":1.643312,"redis_calls":13,"redis_duration_s":0.001411,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014678,"completed_at":"2026-09-14T08:39:09.253Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.410872] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.254Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20c512a7baac8c8af614b7bc","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20c512a7baac8c8af614b7bc: start","job_status":"start","queue_duration_s":1.659363,"scheduling_latency_s":1.65938} gitlab # [ 266.422949] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.269Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"20c512a7baac8c8af614b7bc","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-20c512a7baac8c8af614b7bc: done: 0.014811 sec","job_status":"done","queue_duration_s":1.659363,"scheduling_latency_s":1.65938,"redis_calls":13,"redis_duration_s":0.001666,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000503,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001163,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014811,"completed_at":"2026-09-14T08:39:09.269Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.460537] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.270Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77cae74bba95805f156b2487","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77cae74bba95805f156b2487: start","job_status":"start","queue_duration_s":1.675211,"scheduling_latency_s":1.675227} gitlab # [ 266.472225] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.285Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77cae74bba95805f156b2487","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77cae74bba95805f156b2487: done: 0.014714 sec","job_status":"done","queue_duration_s":1.675211,"scheduling_latency_s":1.675227,"redis_calls":13,"redis_duration_s":0.001405,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000867,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010436,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014714,"completed_at":"2026-09-14T08:39:09.285Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.510406] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.286Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f034aa863495cb3e2c20a20f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f034aa863495cb3e2c20a20f: start","job_status":"start","queue_duration_s":1.691052,"scheduling_latency_s":1.691069} gitlab # [ 266.523342] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f034aa863495cb3e2c20a20f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f034aa863495cb3e2c20a20f: done: 0.014398 sec","job_status":"done","queue_duration_s":1.691052,"scheduling_latency_s":1.691069,"redis_calls":13,"redis_duration_s":0.001398,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000894,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014398,"completed_at":"2026-09-14T08:39:09.300Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.561420] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.302Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79c4a0e654d4e978acd2b26e","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79c4a0e654d4e978acd2b26e: start","job_status":"start","queue_duration_s":1.70711,"scheduling_latency_s":1.707127} gitlab # [ 266.573446] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"79c4a0e654d4e978acd2b26e","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-79c4a0e654d4e978acd2b26e: done: 0.014512 sec","job_status":"done","queue_duration_s":1.70711,"scheduling_latency_s":1.707127,"redis_calls":13,"redis_duration_s":0.00139,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00052,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00087,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010214,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014512,"completed_at":"2026-09-14T08:39:09.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.611557] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ee59c020edba5d8b9e0e642","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ee59c020edba5d8b9e0e642: start","job_status":"start","queue_duration_s":1.722703,"scheduling_latency_s":1.722719} gitlab # [ 266.623576] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0ee59c020edba5d8b9e0e642","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0ee59c020edba5d8b9e0e642: done: 0.014743 sec","job_status":"done","queue_duration_s":1.722703,"scheduling_latency_s":1.722719,"redis_calls":13,"redis_duration_s":0.0014399999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000514,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010422,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014743,"completed_at":"2026-09-14T08:39:09.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.661938] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.333Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14fc5f5b227207f6cc394679","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14fc5f5b227207f6cc394679: start","job_status":"start","queue_duration_s":1.738537,"scheduling_latency_s":1.738553} gitlab # [ 266.674089] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.352Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"14fc5f5b227207f6cc394679","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-14fc5f5b227207f6cc394679: done: 0.018457 sec","job_status":"done","queue_duration_s":1.738537,"scheduling_latency_s":1.738553,"redis_calls":13,"redis_duration_s":0.00522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001073,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.004147,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010644,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018457,"completed_at":"2026-09-14T08:39:09.352Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.712306] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.353Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0411a0c3115c8065fe844f05","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0411a0c3115c8065fe844f05: start","job_status":"start","queue_duration_s":1.758424,"scheduling_latency_s":1.75844} gitlab # [ 266.725014] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.370Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0411a0c3115c8065fe844f05","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0411a0c3115c8065fe844f05: done: 0.016691 sec","job_status":"done","queue_duration_s":1.758424,"scheduling_latency_s":1.75844,"redis_calls":13,"redis_duration_s":0.003382,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000927,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002455,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016691,"completed_at":"2026-09-14T08:39:09.370Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.763269] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.371Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6511d3768d862aee262d963c","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6511d3768d862aee262d963c: start","job_status":"start","queue_duration_s":1.776682,"scheduling_latency_s":1.776699} gitlab # [ 266.775554] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.387Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6511d3768d862aee262d963c","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6511d3768d862aee262d963c: done: 0.015629 sec","job_status":"done","queue_duration_s":1.776682,"scheduling_latency_s":1.776699,"redis_calls":13,"redis_duration_s":0.002791,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000935,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015629,"completed_at":"2026-09-14T08:39:09.387Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.813483] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.389Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b1f2d78e1b2934871b5a696","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b1f2d78e1b2934871b5a696: start","job_status":"start","queue_duration_s":1.793817,"scheduling_latency_s":1.793833} gitlab # [ 266.825337] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.404Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b1f2d78e1b2934871b5a696","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b1f2d78e1b2934871b5a696: done: 0.015262 sec","job_status":"done","queue_duration_s":1.793817,"scheduling_latency_s":1.793833,"redis_calls":13,"redis_duration_s":0.002705,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000874,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001831,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015262,"completed_at":"2026-09-14T08:39:09.404Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.863458] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.406Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"119cb0ace579c611de296eb3","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-119cb0ace579c611de296eb3: start","job_status":"start","queue_duration_s":1.810462,"scheduling_latency_s":1.810477} gitlab # [ 266.875705] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.422Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"119cb0ace579c611de296eb3","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-119cb0ace579c611de296eb3: done: 0.016615 sec","job_status":"done","queue_duration_s":1.810462,"scheduling_latency_s":1.810477,"redis_calls":13,"redis_duration_s":0.002699,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001735,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016615,"completed_at":"2026-09-14T08:39:09.422Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.913521] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd3eaa475808f47e9f2e1d7d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd3eaa475808f47e9f2e1d7d: start","job_status":"start","queue_duration_s":1.828519,"scheduling_latency_s":1.828534} gitlab # [ 266.926550] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.439Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bd3eaa475808f47e9f2e1d7d","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bd3eaa475808f47e9f2e1d7d: done: 0.015657 sec","job_status":"done","queue_duration_s":1.828519,"scheduling_latency_s":1.828534,"redis_calls":13,"redis_duration_s":0.00275,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000944,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001806,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015657,"completed_at":"2026-09-14T08:39:09.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 266.964432] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e77a10edfc646d825b74df","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e77a10edfc646d825b74df: start","job_status":"start","queue_duration_s":1.84559,"scheduling_latency_s":1.845606} gitlab # [ 266.976174] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e9e77a10edfc646d825b74df","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e9e77a10edfc646d825b74df: done: 0.015095 sec","job_status":"done","queue_duration_s":1.84559,"scheduling_latency_s":1.845606,"redis_calls":13,"redis_duration_s":0.002593,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000908,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009695,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015095,"completed_at":"2026-09-14T08:39:09.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.014088] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.457Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b2fc20a2ab11c0c749779a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b2fc20a2ab11c0c749779a: start","job_status":"start","queue_duration_s":1.862009,"scheduling_latency_s":1.862026} gitlab # [ 267.026570] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.473Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b3b2fc20a2ab11c0c749779a","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b3b2fc20a2ab11c0c749779a: done: 0.016416 sec","job_status":"done","queue_duration_s":1.862009,"scheduling_latency_s":1.862026,"redis_calls":13,"redis_duration_s":0.00326,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001921,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016416,"completed_at":"2026-09-14T08:39:09.473Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.064502] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.475Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afcb1df60cd63c5621678f7c","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afcb1df60cd63c5621678f7c: start","job_status":"start","queue_duration_s":1.879836,"scheduling_latency_s":1.879852} gitlab # [ 267.076877] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"afcb1df60cd63c5621678f7c","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-afcb1df60cd63c5621678f7c: done: 0.0155 sec","job_status":"done","queue_duration_s":1.879836,"scheduling_latency_s":1.879852,"redis_calls":13,"redis_duration_s":0.002773,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001825,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009812,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0155,"completed_at":"2026-09-14T08:39:09.490Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.114549] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.491Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e5de37a50129e620ddb8bf","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e5de37a50129e620ddb8bf: start","job_status":"start","queue_duration_s":1.896609,"scheduling_latency_s":1.896627} gitlab # [ 267.127643] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.506Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e5de37a50129e620ddb8bf","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e5de37a50129e620ddb8bf: done: 0.014334 sec","job_status":"done","queue_duration_s":1.896609,"scheduling_latency_s":1.896627,"redis_calls":13,"redis_duration_s":0.001447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000906,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010041,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014334,"completed_at":"2026-09-14T08:39:09.506Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.165591] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f38a2ffd28d961f4ee01e4","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f38a2ffd28d961f4ee01e4: start","job_status":"start","queue_duration_s":1.912006,"scheduling_latency_s":1.912022} gitlab # [ 267.177830] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"19f38a2ffd28d961f4ee01e4","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-19f38a2ffd28d961f4ee01e4: done: 0.01436 sec","job_status":"done","queue_duration_s":1.912006,"scheduling_latency_s":1.912022,"redis_calls":13,"redis_duration_s":0.001462,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009674,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01436,"completed_at":"2026-09-14T08:39:09.521Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.216000] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"384e90f18c7f590bc6954572","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-384e90f18c7f590bc6954572: start","job_status":"start","queue_duration_s":1.927814,"scheduling_latency_s":1.927831} gitlab # [ 267.228096] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"384e90f18c7f590bc6954572","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-384e90f18c7f590bc6954572: done: 0.01481 sec","job_status":"done","queue_duration_s":1.927814,"scheduling_latency_s":1.927831,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000527,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01481,"completed_at":"2026-09-14T08:39:09.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.265684] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a109ee4bf026f0703cdb2afb","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a109ee4bf026f0703cdb2afb: start","job_status":"start","queue_duration_s":1.943784,"scheduling_latency_s":1.9438} gitlab # [ 267.277754] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a109ee4bf026f0703cdb2afb","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a109ee4bf026f0703cdb2afb: done: 0.014297 sec","job_status":"done","queue_duration_s":1.943784,"scheduling_latency_s":1.9438,"redis_calls":13,"redis_duration_s":0.001659,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000505,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014297,"completed_at":"2026-09-14T08:39:09.553Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.315976] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9149b8be1c328b3399f520d5","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9149b8be1c328b3399f520d5: start","job_status":"start","queue_duration_s":1.959143,"scheduling_latency_s":1.95916} gitlab # [ 267.328307] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9149b8be1c328b3399f520d5","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9149b8be1c328b3399f520d5: done: 0.014932 sec","job_status":"done","queue_duration_s":1.959143,"scheduling_latency_s":1.95916,"redis_calls":13,"redis_duration_s":0.001707,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001168,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010406,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014932,"completed_at":"2026-09-14T08:39:09.569Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.367109] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.570Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b0b432cd90e48811aa6dbbd","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b0b432cd90e48811aa6dbbd: start","job_status":"start","queue_duration_s":1.975222,"scheduling_latency_s":1.975239} gitlab # [ 267.379309] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b0b432cd90e48811aa6dbbd","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b0b432cd90e48811aa6dbbd: done: 0.014663 sec","job_status":"done","queue_duration_s":1.975222,"scheduling_latency_s":1.975239,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014663,"completed_at":"2026-09-14T08:39:09.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.417852] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.586Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b1aee69f6a217c6552ba88","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b1aee69f6a217c6552ba88: start","job_status":"start","queue_duration_s":1.990973,"scheduling_latency_s":1.990989} gitlab # [ 267.430038] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"05b1aee69f6a217c6552ba88","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-05b1aee69f6a217c6552ba88: done: 0.0144 sec","job_status":"done","queue_duration_s":1.990973,"scheduling_latency_s":1.990989,"redis_calls":13,"redis_duration_s":0.0014039999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0144,"completed_at":"2026-09-14T08:39:09.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.468815] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca756f24597b728f3341ba3","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca756f24597b728f3341ba3: start","job_status":"start","queue_duration_s":2.006487,"scheduling_latency_s":2.006506} gitlab # [ 267.481052] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ca756f24597b728f3341ba3","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ca756f24597b728f3341ba3: done: 0.014602 sec","job_status":"done","queue_duration_s":2.006487,"scheduling_latency_s":2.006506,"redis_calls":13,"redis_duration_s":0.001459,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000895,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010238,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014602,"completed_at":"2026-09-14T08:39:09.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.519657] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1bfe29c779a1360a9e2d638","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1bfe29c779a1360a9e2d638: start","job_status":"start","queue_duration_s":2.022226,"scheduling_latency_s":2.022243} gitlab # [ 267.532152] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1bfe29c779a1360a9e2d638","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1bfe29c779a1360a9e2d638: done: 0.014608 sec","job_status":"done","queue_duration_s":2.022226,"scheduling_latency_s":2.022243,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000878,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010103,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014608,"completed_at":"2026-09-14T08:39:09.631Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.571924] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ec31b3aca0e8815494e0205","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ec31b3aca0e8815494e0205: start","job_status":"start","queue_duration_s":2.037916,"scheduling_latency_s":2.037934} gitlab # [ 267.584490] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7ec31b3aca0e8815494e0205","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7ec31b3aca0e8815494e0205: done: 0.015486 sec","job_status":"done","queue_duration_s":2.037916,"scheduling_latency_s":2.037934,"redis_calls":13,"redis_duration_s":0.0024850000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000523,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015486,"completed_at":"2026-09-14T08:39:09.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.623264] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"accf1703243ee7ba2ec4b130","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-accf1703243ee7ba2ec4b130: start","job_status":"start","queue_duration_s":2.054536,"scheduling_latency_s":2.054553} gitlab # [ 267.635682] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"accf1703243ee7ba2ec4b130","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-accf1703243ee7ba2ec4b130: done: 0.014421 sec","job_status":"done","queue_duration_s":2.054536,"scheduling_latency_s":2.054553,"redis_calls":13,"redis_duration_s":0.001383,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000502,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010145,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014421,"completed_at":"2026-09-14T08:39:09.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.674441] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.665Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b8f5433d384919c5cc41871","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b8f5433d384919c5cc41871: start","job_status":"start","queue_duration_s":2.070421,"scheduling_latency_s":2.070438} gitlab # [ 267.686931] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.679Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b8f5433d384919c5cc41871","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b8f5433d384919c5cc41871: done: 0.01421 sec","job_status":"done","queue_duration_s":2.070421,"scheduling_latency_s":2.070438,"redis_calls":13,"redis_duration_s":0.001395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000526,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01421,"completed_at":"2026-09-14T08:39:09.679Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.725666] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a277d6eb592fbe6e7c0c1d0b","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a277d6eb592fbe6e7c0c1d0b: start","job_status":"start","queue_duration_s":2.085747,"scheduling_latency_s":2.085764} gitlab # [ 267.738657] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.695Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a277d6eb592fbe6e7c0c1d0b","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a277d6eb592fbe6e7c0c1d0b: done: 0.014562 sec","job_status":"done","queue_duration_s":2.085747,"scheduling_latency_s":2.085764,"redis_calls":13,"redis_duration_s":0.001469,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000915,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014562,"completed_at":"2026-09-14T08:39:09.695Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.777834] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.696Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8b87b509ba99891f9ae409f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8b87b509ba99891f9ae409f: start","job_status":"start","queue_duration_s":2.101409,"scheduling_latency_s":2.101426} gitlab # [ 267.790257] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.710Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a8b87b509ba99891f9ae409f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a8b87b509ba99891f9ae409f: done: 0.014002 sec","job_status":"done","queue_duration_s":2.101409,"scheduling_latency_s":2.101426,"redis_calls":13,"redis_duration_s":0.001376,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00088,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014002,"completed_at":"2026-09-14T08:39:09.710Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.829561] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66ef972875c841de42aa7fb0","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66ef972875c841de42aa7fb0: start","job_status":"start","queue_duration_s":2.116476,"scheduling_latency_s":2.116493} gitlab # [ 267.866317] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.726Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"66ef972875c841de42aa7fb0","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-66ef972875c841de42aa7fb0: done: 0.014363 sec","job_status":"done","queue_duration_s":2.116476,"scheduling_latency_s":2.116493,"redis_calls":13,"redis_duration_s":0.0013809999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000495,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014363,"completed_at":"2026-09-14T08:39:09.725Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.937826] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.727Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a06c72d4386bc6c08bf66c0","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a06c72d4386bc6c08bf66c0: start","job_status":"start","queue_duration_s":2.131996,"scheduling_latency_s":2.132012} gitlab # [ 267.952887] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a06c72d4386bc6c08bf66c0","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a06c72d4386bc6c08bf66c0: done: 0.014241 sec","job_status":"done","queue_duration_s":2.131996,"scheduling_latency_s":2.132012,"redis_calls":13,"redis_duration_s":0.001429,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014241,"completed_at":"2026-09-14T08:39:09.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 267.999865] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01858b94dd31de5338e0e9ba","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01858b94dd31de5338e0e9ba: start","job_status":"start","queue_duration_s":2.147332,"scheduling_latency_s":2.147351} gitlab # [ 268.013257] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.756Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01858b94dd31de5338e0e9ba","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01858b94dd31de5338e0e9ba: done: 0.014418 sec","job_status":"done","queue_duration_s":2.147332,"scheduling_latency_s":2.147351,"redis_calls":13,"redis_duration_s":0.001409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014418,"completed_at":"2026-09-14T08:39:09.756Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.051775] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8337ccbe2dfebe208e2cac5f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8337ccbe2dfebe208e2cac5f: start","job_status":"start","queue_duration_s":2.16279,"scheduling_latency_s":2.162807} gitlab # [ 268.064021] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.772Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8337ccbe2dfebe208e2cac5f","created_at":"2026-09-14T08:39:07.064Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-7c91c8570e4b4684","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"480bdf3a1f7e4ed4c7a28c54899d7b6f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.595Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8337ccbe2dfebe208e2cac5f: done: 0.014076 sec","job_status":"done","queue_duration_s":2.16279,"scheduling_latency_s":2.162807,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009733,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014076,"completed_at":"2026-09-14T08:39:09.771Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.102247] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:09.773Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d39579d2e93a1cce5c451a85","created_at":"2026-09-14T08:39:07.594Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c693d59e1d71b0c5f21efce893980634","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.607Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d39579d2e93a1cce5c451a85: start","job_status":"start","queue_duration_s":2.165602,"scheduling_latency_s":2.165619} gitlab # [ 268.116205] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.090Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"d39579d2e93a1cce5c451a85","created_at":"2026-09-14T08:39:07.594Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c693d59e1d71b0c5f21efce893980634","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.607Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-d39579d2e93a1cce5c451a85: done: 0.317321 sec","job_status":"done","queue_duration_s":2.165602,"scheduling_latency_s":2.165619,"redis_calls":562,"redis_duration_s":0.055327,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.054627,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.29702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.317321,"completed_at":"2026-09-14T08:39:10.090Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.155439] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.091Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"70f9d63b7f70ee77b8431044","created_at":"2026-09-14T08:39:07.732Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b87fb418c03cefe92ef2fcb1538987d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.741Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-70f9d63b7f70ee77b8431044: start","job_status":"start","queue_duration_s":2.350761,"scheduling_latency_s":2.350783} gitlab # [ 268.170871] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.099Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 268.174247] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.104Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"70f9d63b7f70ee77b8431044","created_at":"2026-09-14T08:39:07.732Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b87fb418c03cefe92ef2fcb1538987d0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.741Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-70f9d63b7f70ee77b8431044: done: 0.012722 sec","job_status":"done","queue_duration_s":2.350761,"scheduling_latency_s":2.350783,"redis_calls":7,"redis_duration_s":0.000926,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000661,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000265,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012722,"completed_at":"2026-09-14T08:39:10.104Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.214040] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.105Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"45b2db846130ba974eeae175","created_at":"2026-09-14T08:39:07.759Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47cf7b24fae77d871ca6b57d7523498d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.771Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-45b2db846130ba974eeae175: start","job_status":"start","queue_duration_s":2.33388,"scheduling_latency_s":2.333897} gitlab # [ 268.228178] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.128Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"45b2db846130ba974eeae175","created_at":"2026-09-14T08:39:07.759Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"47cf7b24fae77d871ca6b57d7523498d","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.771Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-45b2db846130ba974eeae175: done: 0.022989 sec","job_status":"done","queue_duration_s":2.33388,"scheduling_latency_s":2.333897,"redis_calls":10,"redis_duration_s":0.0013909999999999999,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000807,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000584,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014458,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00775228,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.022989,"completed_at":"2026-09-14T08:39:10.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004512,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 268.269602] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.129Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"caeeb6e59585c01748666535","created_at":"2026-09-14T08:39:07.852Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"622437934a0f25b3b6facaaf1d7ca46f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.863Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-caeeb6e59585c01748666535: start","job_status":"start","queue_duration_s":2.266283,"scheduling_latency_s":2.266301} gitlab # [ 268.283863] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.179Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"caeeb6e59585c01748666535","created_at":"2026-09-14T08:39:07.852Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-1e7ab219b28545b7","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"622437934a0f25b3b6facaaf1d7ca46f","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-09-14T08:39:07.863Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-caeeb6e59585c01748666535: done: 0.049454 sec","job_status":"done","queue_duration_s":2.266283,"scheduling_latency_s":2.266301,"redis_calls":9,"redis_duration_s":0.001458,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000753,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000705,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.025,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.025,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019859,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005238,"exclusive_lock_hold_duration_s":0.02622068,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.049454,"completed_at":"2026-09-14T08:39:10.179Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.325906] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.180Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"862d30f830f8da6ec048fc0f","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-862d30f830f8da6ec048fc0f: start","job_status":"start","queue_duration_s":1.424336,"scheduling_latency_s":1.424355} gitlab # [ 268.337974] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.193Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.339598] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.202Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"862d30f830f8da6ec048fc0f","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-862d30f830f8da6ec048fc0f: done: 0.021581 sec","job_status":"done","queue_duration_s":1.424336,"scheduling_latency_s":1.424355,"redis_calls":10,"redis_duration_s":0.001462,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000647,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021581,"completed_at":"2026-09-14T08:39:10.202Z","load_balancing_strategy":"primary","db_duration_s":0.002839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.378089] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.203Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d6423092514f4ec641391fe3","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d6423092514f4ec641391fe3: start","job_status":"start","queue_duration_s":1.447095,"scheduling_latency_s":1.447116} gitlab # [ 268.390193] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.214Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.391809] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.223Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d6423092514f4ec641391fe3","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d6423092514f4ec641391fe3: done: 0.019961 sec","job_status":"done","queue_duration_s":1.447095,"scheduling_latency_s":1.447116,"redis_calls":10,"redis_duration_s":0.001373,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000772,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01405,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019961,"completed_at":"2026-09-14T08:39:10.223Z","load_balancing_strategy":"primary","db_duration_s":0.00243,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.429383] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.224Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"548307d7c6af48759f909931","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-548307d7c6af48759f909931: start","job_status":"start","queue_duration_s":1.468248,"scheduling_latency_s":1.468267} gitlab # [ 268.441148] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.236Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.442773] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.245Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"548307d7c6af48759f909931","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-548307d7c6af48759f909931: done: 0.02082 sec","job_status":"done","queue_duration_s":1.468248,"scheduling_latency_s":1.468267,"redis_calls":10,"redis_duration_s":0.001617,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001062,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02082,"completed_at":"2026-09-14T08:39:10.245Z","load_balancing_strategy":"primary","db_duration_s":0.003384,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 268.480267] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.246Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66e4e703b601cd2cd8d659e0","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66e4e703b601cd2cd8d659e0: start","job_status":"start","queue_duration_s":1.490132,"scheduling_latency_s":1.490151} gitlab # [ 268.492189] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.257Z","message":"No existing merge request to be cleaned up."} gitlab # [ 268.493773] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:39:10.265Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"66e4e703b601cd2cd8d659e0","created_at":"2026-09-14T08:39:08.727Z","trace_propagation_headers":{"sentry-trace":"3745c551bfd64e9dad5a1e4e2ed2c2fb-b7361274a73a47dd","baggage":"sentry-trace_id=3745c551bfd64e9dad5a1e4e2ed2c2fb,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"a1da3bdcc78001f2af011d7360c39ff6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:39:08.756Z","job_size_bytes":2,"pid":2020,"sidekiq_tid":"maoc","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-66e4e703b601cd2cd8d659e0: done: 0.019106 sec","job_status":"done","queue_duration_s":1.490132,"scheduling_latency_s":1.490151,"redis_calls":10,"redis_duration_s":0.001278,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000727,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019106,"completed_at":"2026-09-14T08:39:10.265Z","load_balancing_strategy":"primary","db_duration_s":0.00262,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 269.199670] setup-gitlab-start[2938]: W, [2026-09-14T08:39:13.026797 #2938] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 309.218765] redis[788]: 10 changes in 300 seconds. Saving... gitlab # [ 309.225666] redis[788]: Background saving started by pid 3288 gitlab # [ 309.229663] redis[3288]: BGSAVE done, 444 keys saved, 0 keys skipped, 115938 bytes written. gitlab # [ 309.234005] redis[3288]: DB saved on disk gitlab # [ 309.236433] redis[3288]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 309.320698] redis[788]: Background saving terminated with success gitlab # [ 311.055503] postgres[866]: [866] LOG: checkpoint starting: time gitlab # [ 311.462962] postgres[3273]: [3273] ERROR: relation "table_without_models" does not exist gitlab # [ 311.466572] postgres[3273]: [3273] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 311.906903] gitaly[897]: time="2026-09-14T08:39:55.733Z" level=info msg="burden monitor stats" pid=897 gitlab # [ 318.015115] runuser[2937]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 318.017189] setup-gitlab-start[2915]: + systemctl restart gitlab-sidekiq.service gitlab # [ 318.048056] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 318.049395] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:01.875Z","message":"Shutting down"} gitlab # [ 318.051884] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:01.879Z","message":"Scheduler exiting..."} gitlab # [ 318.056353] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:01.879Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 318.061479] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:01.880Z","message":"Scheduler exiting..."} gitlab # [ 318.557788] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:02.385Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 322.062688] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:05.889Z","message":"Bye!"} gitlab # [ 322.105917] sidekiq-cluster[2020]: {"severity":"INFO","time":"2026-09-14T08:40:05.933Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2020,"worker_id":"sidekiq_0","memwd_rss_bytes":922648576,"retry":0} gitlab # [ 323.288745] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 323.289172] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 323.289399] systemd[1]: gitlab-sidekiq.service: Consumed 1min 46.892s CPU time over 3min 20.202s wall clock time, 937.1M memory peak, 4.5M read from disk. gitlab # [ 323.302518] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 323.306943] setup-gitlab-start[2915]: + mkdir -p /var/lib/secrets gitlab # [ 323.311737] setup-gitlab-start[2915]: + echo nixbot-test-token-of-20-chars gitlab # [ 323.314768] setup-gitlab-start[2915]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 323.317844] setup-gitlab-start[2915]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars gitlab # [ 323.951568] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:07 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 590 gitlab # [ 325.204304] gitaly[897]: time="2026-09-14T08:40:09.031Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.031" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.221816] gitaly[897]: time="2026-09-14T08:40:09.050Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.031" grpc.time_ms=18.66 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.484763] gitaly[897]: time="2026-09-14T08:40:09.312Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.291" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.312" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.516857] gitaly[897]: time="2026-09-14T08:40:09.313Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.291" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.312" grpc.time_ms=0.585 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.602727] gitaly[897]: time="2026-09-14T08:40:09.430Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR diskcache=4d2ebd4a-1e0c-483c-9251-a9a8537aa1db grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T08:40:19.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.359" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.632901] gitaly[897]: time="2026-09-14T08:40:09.431Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=40 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=44 command.spawn_token_fork_ms=6 command.system_time_ms=36 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T08:40:19.008" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.359" grpc.time_ms=71.715 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.659685] gitaly[897]: time="2026-09-14T08:40:09.454Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.102" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.453" pid=897 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 325.676746] gitaly[897]: time="2026-09-14T08:40:09.454Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.102" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.453" grpc.time_ms=1.439 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.695230] gitaly[897]: time="2026-09-14T08:40:09.480Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:19.112" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.463" grpc.time_ms=16.636 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.760141] gitaly[897]: time="2026-09-14T08:40:09.588Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.236" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.587" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.772789] gitaly[897]: time="2026-09-14T08:40:09.589Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.236" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.587" grpc.time_ms=0.772 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.821875] gitaly[897]: time="2026-09-14T08:40:09.650Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR diskcache=f48768c0-6da1-4ea2-8d63-d83a7222854a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T08:40:19.253" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.604" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.844401] gitaly[897]: time="2026-09-14T08:40:09.650Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=22 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1437 command.oublock=32 command.real_time_ms=25 command.spawn_token_fork_ms=5 command.system_time_ms=22 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-09-14T08:40:19.253" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.604" grpc.time_ms=46.211 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.865746] gitaly[897]: time="2026-09-14T08:40:09.662Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.661" pid=897 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 325.879420] gitaly[897]: time="2026-09-14T08:40:09.662Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.661" grpc.time_ms=0.995 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.892161] gitaly[897]: time="2026-09-14T08:40:09.685Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:19.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.669" grpc.time_ms=15.72 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.919565] gitaly[897]: time="2026-09-14T08:40:09.747Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:40:19.095" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.746" grpc.time_ms=0.4 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 325.936251] gitaly[897]: time="2026-09-14T08:40:09.762Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T08:40:19.110" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:09.761" grpc.time_ms=0.743 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 326.837383] gitaly[897]: time="2026-09-14T08:40:10.664Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5PC2G8YM43N0HARYX1GR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:20.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:10.647" grpc.time_ms=16.747 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 327.525569] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:11 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 3542 gitlab # [ 327.529962] setup-gitlab-start[3363]: {"id":1,"message":null,"error":null} gitlab # [ 328.260125] sidekiq-cluster[3346]: {"severity":"INFO","time":"2026-09-14T08:40:12.052Z","message":"Starting cluster with 1 processes"} gitlab # [ 332.774456] gitaly[897]: time="2026-09-14T08:40:16.601Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH5YRBHZRFGVQ5WZK36P1Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:26.233" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:16.584" grpc.time_ms=17.094 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.902035] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:16 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 333 gitlab # [ 332.908043] setup-gitlab-start[2915]: + mkdir -p /tmp/test-flake gitlab # [ 332.912862] setup-gitlab-start[2915]: + cd /tmp/test-flake gitlab # [ 332.914423] setup-gitlab-start[2915]: + git init -b master gitlab # [ 332.939483] setup-gitlab-start[3442]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 332.943920] setup-gitlab-start[2915]: + git config user.name test gitlab # [ 332.951595] setup-gitlab-start[2915]: + git config user.email test@example.com gitlab # [ 332.958639] setup-gitlab-start[2915]: + cat gitlab # [ 332.964407] setup-gitlab-start[2915]: + git add flake.nix gitlab # [ 332.973106] setup-gitlab-start[2915]: + git commit -m 'initial commit' gitlab # [ 333.000203] setup-gitlab-start[3447]: [master (root-commit) 28fc4ed] initial commit gitlab # [ 333.003287] setup-gitlab-start[3447]: 1 file changed, 10 insertions(+) gitlab # [ 333.007639] setup-gitlab-start[3447]: create mode 100644 flake.nix gitlab # [ 333.010408] setup-gitlab-start[2915]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 333.018557] setup-gitlab-start[2915]: + git push -u origin master gitlab # [ 333.129478] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:16 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 63 gitlab # [ 333.883389] gitaly[897]: time="2026-09-14T08:40:17.710Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH5ZA4THB7F6VGSKWJD7T4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:17.691" grpc.time_ms=18.945 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 333.924378] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 747 gitlab # [ 334.659708] gitaly[897]: time="2026-09-14T08:40:18.487Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:28.003" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:18.473" grpc.time_ms=14.175 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.817962] gitaly[897]: time="2026-09-14T08:40:18.646Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-14T08:40:48.024" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-14T08:40:18.624" grpc.time_ms=20.907 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.934971] gitaly[897]: time="2026-09-14T08:40:18.762Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 duration_ms=651ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 335.054365] gitaly[897]: time="2026-09-14T08:40:18.882Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 duration_ms=119ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 335.068119] gitaly[897]: time="2026-09-14T08:40:18.883Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:18.108" grpc.time_ms=774.648 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.160735] gitaly[897]: time="2026-09-14T08:40:18.988Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:18.986" grpc.time_ms=1.755 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.210180] gitaly[897]: time="2026-09-14T08:40:19.038Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:19.037" grpc.time_ms=0.98 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.257161] gitaly[897]: time="2026-09-14T08:40:19.085Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:19.083" grpc.time_ms=1.735 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.441660] gitaly[897]: time="2026-09-14T08:40:19.269Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:40:29.231" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:19.252" grpc.time_ms=17.204 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.500934] gitaly[897]: time="2026-09-14T08:40:19.329Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=32 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1432 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=4 command.system_time_ms=32 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-14T08:40:29.266" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:19.287" grpc.time_ms=41.143 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.539874] gitaly[897]: time="2026-09-14T08:40:19.368Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 duration_ms=229ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 335.549691] gitaly[897]: time="2026-09-14T08:40:19.369Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:19.137" grpc.time_ms=231.993 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.576577] gitaly[897]: time="2026-09-14T08:40:19.404Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 diskcache=63f4b27e-fb8d-40a0-a6d7-f0f347786ee1 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:17.991" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.594080] gitaly[897]: time="2026-09-14T08:40:19.405Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=352 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13718 command.oublock=32 command.real_time_ms=1403 command.spawn_token_fork_ms=2 command.system_time_ms=200 command.user_time_ms=152 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=685 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:17.991" grpc.time_ms=1413.657 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.615450] setup-gitlab-start[3451]: To http://gitlab/root/test-flake.git gitlab # [ 335.616740] setup-gitlab-start[3451]: * [new branch] master -> master gitlab # [ 335.617960] setup-gitlab-start[3451]: branch 'master' set up to track 'origin/master'. gitlab # [ 335.619777] gitlab-workhorse[820]: time="2026-09-14T08:40:19Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 335.622264] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:19 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1676 gitlab # [ 335.624933] systemd[1]: Finished setup-gitlab.service. gitlab # [ 335.625924] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 336.83 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 336.914282] nixbot[3543]: {"ts": "2026-09-14T08:40:20.741Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 336.941529] nixbot[3543]: {"ts": "2026-09-14T08:40:20.769Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 336.959070] nixbot[3543]: {"ts": "2026-09-14T08:40:20.787Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 336.965766] nixbot[3543]: {"ts": "2026-09-14T08:40:20.792Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 336.972614] nixbot[3543]: {"ts": "2026-09-14T08:40:20.796Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 336.978914] nixbot[3543]: {"ts": "2026-09-14T08:40:20.801Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 336.985414] nixbot[3543]: {"ts": "2026-09-14T08:40:20.806Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 336.991957] nixbot[3543]: {"ts": "2026-09-14T08:40:20.808Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 336.998289] nixbot[3543]: {"ts": "2026-09-14T08:40:20.814Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 337.004827] nixbot[3543]: {"ts": "2026-09-14T08:40:20.816Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 337.011660] nixbot[3543]: {"ts": "2026-09-14T08:40:20.820Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 337.018091] nixbot[3543]: {"ts": "2026-09-14T08:40:20.826Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 337.025210] nixbot[3543]: {"ts": "2026-09-14T08:40:20.829Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 337.032208] nixbot[3543]: {"ts": "2026-09-14T08:40:20.831Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 337.039259] nixbot[3543]: {"ts": "2026-09-14T08:40:20.835Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 337.045452] nixbot[3543]: {"ts": "2026-09-14T08:40:20.837Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 337.050923] nixbot[3543]: {"ts": "2026-09-14T08:40:20.839Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 337.056621] nixbot[3543]: {"ts": "2026-09-14T08:40:20.842Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 337.061937] nixbot[3543]: {"ts": "2026-09-14T08:40:20.847Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 337.067266] nixbot[3543]: {"ts": "2026-09-14T08:40:20.849Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 337.072577] nixbot[3543]: {"ts": "2026-09-14T08:40:20.850Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 337.077553] nixbot[3543]: {"ts": "2026-09-14T08:40:20.855Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 337.082344] nixbot[3543]: {"ts": "2026-09-14T08:40:20.857Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 337.086965] nixbot[3543]: {"ts": "2026-09-14T08:40:20.859Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 337.092520] nixbot[3543]: {"ts": "2026-09-14T08:40:20.861Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 337.096258] nixbot[3543]: {"ts": "2026-09-14T08:40:20.863Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"} gitlab # [ 337.099767] nixbot[3543]: {"ts": "2026-09-14T08:40:20.865Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"} gitlab # [ 337.103286] nixbot[3543]: {"ts": "2026-09-14T08:40:20.867Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"} gitlab # [ 337.107057] nixbot[3543]: {"ts": "2026-09-14T08:40:20.868Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"} gitlab # [ 337.117342] nixbot[3543]: {"ts": "2026-09-14T08:40:20.878Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"} gitlab # [ 337.122902] nixbot[3543]: {"ts": "2026-09-14T08:40:20.882Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"} gitlab # [ 337.126240] nixbot[3543]: {"ts": "2026-09-14T08:40:20.884Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"} gitlab # [ 337.129631] nixbot[3543]: {"ts": "2026-09-14T08:40:20.886Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"} gitlab # [ 337.133139] nixbot[3543]: {"ts": "2026-09-14T08:40:20.891Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"} gitlab # [ 337.136501] nixbot[3543]: {"ts": "2026-09-14T08:40:20.892Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"} gitlab # [ 337.141064] nixbot[3543]: {"ts": "2026-09-14T08:40:20.897Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"} gitlab # [ 337.567432] nixbot[3543]: INFO: Started server process [3543] gitlab # [ 337.570454] nixbot[3543]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 337.590540] nixbot[3543]: INFO: ::ffff:127.0.0.1:45966 - "GET /health HTTP/1.1" 200 OK gitlab # [ 337.596963] systemd[1]: Started nixbot CI service. gitlab # [ 337.599031] systemd[1]: Reached target Multi-User System. gitlab # [ 337.599916] systemd[1]: Startup finished in 772ms (kernel) + 4.175s (initrd) + 5min 32.651s (userspace) = 5min 37.599s. gitlab: (finished: waiting for unit nixbot.service, in 2.14 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 338.567195] nixbot[3543]: INFO: ::ffff:127.0.0.1:45968 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.21 seconds) gitlab # [ 338.773912] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:22 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 146 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 339.919447] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:23 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 2329 gitlab # [ 339.920775] nixbot[3543]: {"ts": "2026-09-14T08:40:23.748Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 339.942921] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 111 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.17 seconds) gitlab # [ 339.968034] nixbot[3543]: {"ts": "2026-09-14T08:40:23.796Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 340.109131] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:23 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 127 gitlab # [ 340.113037] nixbot[3543]: {"ts": "2026-09-14T08:40:23.939Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 340.117201] nixbot[3543]: {"ts": "2026-09-14T08:40:23.943Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 340.273633] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 153 gitlab # [ 340.275957] nixbot[3543]: {"ts": "2026-09-14T08:40:24.104Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 340.460790] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 171 gitlab # [ 340.463221] nixbot[3543]: {"ts": "2026-09-14T08:40:24.291Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 340.534073] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 55 gitlab # [ 340.544224] nixbot[3543]: {"ts": "2026-09-14T08:40:24.363Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 340.724125] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 185 gitlab # [ 340.735902] nixbot[3543]: {"ts": "2026-09-14T08:40:24.560Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab # [ 340.880979] gitaly[897]: time="2026-09-14T08:40:24.706Z" level=info msg="finished unary call with code OK" catfile.duration_ms=6 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=6 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH66QS68AMBZN3DNBAX4PS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-09-14T08:40:54.077" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:24.678" grpc.time_ms=27.097 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.921852] gitaly[897]: time="2026-09-14T08:40:24.733Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH66QS68AMBZN3DNBAX4PS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:40:54.131" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:24.731" grpc.time_ms=1.272 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 340.943417] gitaly[897]: time="2026-09-14T08:40:24.753Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH66QS68AMBZN3DNBAX4PS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-09-14T08:40:34.268" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:24.739" grpc.time_ms=14.266 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.971536] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 229 gitlab # [ 340.976456] nixbot[3543]: {"ts": "2026-09-14T08:40:24.800Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 341.084740] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 83 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.14 seconds) (finished: subtest: project discovered and webhook registered, in 2.52 seconds) subtest: push delivers the webhook gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails' gitlab # [ 341.107599] nixbot[3543]: {"ts": "2026-09-14T08:40:24.935Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 341.113666] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:24 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 130 gitlab # [ 341.121943] nixbot[3543]: {"ts": "2026-09-14T08:40:24.949Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 341.271858] gitaly[897]: time="2026-09-14T08:40:25.100Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2FH675EZ69D2NEMY5EMN72Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.084" grpc.time_ms=15.524 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.309659] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 94 gitlab # [ 341.439712] gitaly[897]: time="2026-09-14T08:40:25.267Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH678GFP2ZTPWY2SYGR763 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.250" pid=897 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 341.470596] gitaly[897]: time="2026-09-14T08:40:25.268Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH678GFP2ZTPWY2SYGR763 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.250" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.500304] gitlab-workhorse[820]: time="2026-09-14T08:40:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab # [ 341.503549] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 165 gitlab # [ 341.506887] gitaly[897]: time="2026-09-14T08:40:25.269Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=587 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH678GFP2ZTPWY2SYGR763 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.250" grpc.time_ms=17.858 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.536691] gitaly[897]: time="2026-09-14T08:40:25.364Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67DXFXCBW83APX6YJR1T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.345" pid=897 remote_ip="2001:db8:1::1" request_sha=35e02fe91aa34dd4b32c2fef0b148a785486e76b response_bytes=224 span.kind=server system=grpc gitlab # [ 341.551034] gitaly[897]: time="2026-09-14T08:40:25.365Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67DXFXCBW83APX6YJR1T grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.345" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.569887] gitlab-workhorse[820]: time="2026-09-14T08:40:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=224 gitlab # [ 341.572408] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 224 "" "git/2.55.0" 88 gitlab # [ 341.574943] gitaly[897]: time="2026-09-14T08:40:25.365Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1137 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67DXFXCBW83APX6YJR1T grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.345" grpc.time_ms=20.236 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.661017] gitaly[897]: time="2026-09-14T08:40:25.489Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH67J76M8S9ZQE2DEEM03G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.474" grpc.time_ms=14.219 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.680862] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 75 gitlab # [ 341.804296] gitaly[897]: time="2026-09-14T08:40:25.632Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67MNJGAB6GPDNNDF0APJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.613" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 341.834018] gitaly[897]: time="2026-09-14T08:40:25.632Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67MNJGAB6GPDNNDF0APJ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.613" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 341.863787] gitlab-workhorse[820]: time="2026-09-14T08:40:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 341.867766] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 139 gitlab # [ 341.871727] gitaly[897]: time="2026-09-14T08:40:25.633Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH67MNJGAB6GPDNNDF0APJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.613" grpc.time_ms=19.195 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 342.151268] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 249 gitlab # [ 342.153947] gitaly[897]: time="2026-09-14T08:40:25.979Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=28 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH67W0YEWHE0KZCF9B7BX3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=200 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:25.947" grpc.time_ms=29.6 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 342.268173] gitaly[897]: time="2026-09-14T08:40:26.096Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH683YGN0P6J3RVN17ZEC4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:26.061" pid=897 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 342.279680] gitaly[897]: time="2026-09-14T08:40:26.096Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH683YGN0P6J3RVN17ZEC4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:26.061" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 342.296223] gitlab-workhorse[820]: time="2026-09-14T08:40:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 342.298277] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 115 gitlab # [ 342.300399] gitaly[897]: time="2026-09-14T08:40:26.097Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=27 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=2 command.system_time_ms=16 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH683YGN0P6J3RVN17ZEC4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:26.061" grpc.time_ms=35.548 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 342.419280] gitaly[897]: time="2026-09-14T08:40:26.247Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:40:56.045" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:26.246" grpc.time_ms=1.001 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.511881] gitaly[897]: time="2026-09-14T08:40:26.340Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-14T08:40:56.121" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:26.321" grpc.time_ms=18.051 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.564647] gitaly[897]: time="2026-09-14T08:40:26.392Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:40:36.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:26.375" grpc.time_ms=16.844 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.602739] gitaly[897]: time="2026-09-14T08:40:26.431Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:40:36.044" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:26.414" grpc.time_ms=16.114 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.777123] gitaly[897]: time="2026-09-14T08:40:26.605Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T08:40:36.133" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:26.604" grpc.time_ms=0.655 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.812687] gitaly[897]: time="2026-09-14T08:40:26.640Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T08:40:36.152" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:26.623" grpc.time_ms=17.135 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 342.892154] gitaly[897]: time="2026-09-14T08:40:26.717Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:26.715" grpc.time_ms=2.391 pid=897 span.kind=server system=grpc gitlab # [ 342.969301] gitaly[897]: time="2026-09-14T08:40:26.797Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:40:26.796" grpc.time_ms=1.164 pid=897 span.kind=server system=grpc gitlab # [ 343.006659] gitaly[897]: time="2026-09-14T08:40:26.834Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB diskcache=daaae037-a9f9-4d1f-871b-fc53ff575efa grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:40:36.174" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:26.645" pid=897 ref=refs/keep-around/28fc4ed78983aa2a5da210e428415bbec71b479a remote_ip="2001:db8:1::1" revision=28fc4ed78983aa2a5da210e428415bbec71b479a span.kind=server system=grpc username=root gitlab # [ 343.031199] gitaly[897]: time="2026-09-14T08:40:26.835Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=140 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5497 command.oublock=8 command.real_time_ms=181 command.spawn_token_fork_ms=2 command.system_time_ms=83 command.user_time_ms=57 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:40:36.174" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:40:26.645" grpc.time_ms=189.464 pid=897 ref=refs/keep-around/28fc4ed78983aa2a5da210e428415bbec71b479a remote_ip="2001:db8:1::1" revision=28fc4ed78983aa2a5da210e428415bbec71b479a span.kind=server system=grpc username=root gitlab # [ 343.055361] gitaly[897]: time="2026-09-14T08:40:26.845Z" level=info msg="finished unary call with code OK" catfile.duration_ms=2 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=2 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:40:56.241" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:26.842" grpc.time_ms=2.767 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 343.316549] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:27 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 972 gitlab # [ 343.325670] nixbot[3543]: {"ts": "2026-09-14T08:40:27.145Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 343.334649] nixbot[3543]: {"ts": "2026-09-14T08:40:27.160Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-28fc4ed78983-0"} gitlab # [ 343.617427] nixbot[3543]: {"ts": "2026-09-14T08:40:27.445Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-28fc4ed78983-0", "argv0": "bwrap"} gitlab # [ 343.799491] systemd[1]: Started Nix Daemon. gitlab # [ 343.908545] nix-daemon[3720]: accepted connection from pid 3715, user nixbot gitlab # [ 349.047850] nix-daemon[3720]: accepted connection from pid 3769, user nixbot gitlab # [ 349.069296] nix-daemon[3720]: accepted connection from pid 3715, user nixbot gitlab # [ 353.887166] nix-daemon[3720]: accepted connection from pid 3715, user nixbot gitlab # [ 354.112991] gitaly[897]: time="2026-09-14T08:40:37.940Z" level=info msg="finished unary call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 command.spawn_token_fork_ms=1 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6KNJ5SCJS043RNYA0AJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:41:07.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:37.930" grpc.time_ms=9.823 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 354.407029] gitaly[897]: time="2026-09-14T08:40:38.235Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6KNJ5SCJS043RNYA0AJD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:40:48.195" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:38.216" grpc.time_ms=18.608 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 354.585842] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 676 "" "python-httpx/0.28.1" 601 gitlab # [ 354.595785] nixbot[3543]: {"ts": "2026-09-14T08:40:38.414Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 354.663358] gitaly[897]: time="2026-09-14T08:40:38.491Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6M8PRN1M7BRV4M6KFBP7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:41:08.289" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:38.490" grpc.time_ms=1.141 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 354.719885] gitaly[897]: time="2026-09-14T08:40:38.548Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6M8PRN1M7BRV4M6KFBP7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:40:48.063" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:38.533" grpc.time_ms=13.88 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 354.802480] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 207 gitlab # [ 354.810419] nixbot[3543]: {"ts": "2026-09-14T08:40:38.632Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 354.889723] gitaly[897]: time="2026-09-14T08:40:38.717Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6MFNAPDMX5DHWEH1859Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:41:08.114" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:40:38.715" grpc.time_ms=1.624 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 354.966764] gitaly[897]: time="2026-09-14T08:40:38.794Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6MFNAPDMX5DHWEH1859Z grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:40:48.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:40:38.777" grpc.time_ms=17.447 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 355.100982] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 282 gitlab # [ 355.109003] nixbot[3543]: {"ts": "2026-09-14T08:40:38.931Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 355.767845] gitaly[897]: time="2026-09-14T08:40:39.594Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6N9H2H75JZAZ0AR61EWC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.574" grpc.time_ms=19.292 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 355.769229] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 120 gitlab # [ 355.867443] gitaly[897]: time="2026-09-14T08:40:39.695Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NDDW4P11895W6GDK09N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.677" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 355.896804] gitaly[897]: time="2026-09-14T08:40:39.695Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NDDW4P11895W6GDK09N grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.677" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 355.924552] gitlab-workhorse[820]: time="2026-09-14T08:40:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 355.927524] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 99 gitlab # [ 355.930560] gitaly[897]: time="2026-09-14T08:40:39.696Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NDDW4P11895W6GDK09N grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.677" grpc.time_ms=18.85 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 356.028781] gitaly[897]: time="2026-09-14T08:40:39.856Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6NJAYQ3301G0NY7JYVNG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.847" grpc.time_ms=9.437 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 356.057413] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:39 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 102 gitlab # [ 356.102718] gitaly[897]: time="2026-09-14T08:40:39.930Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NNJZWCCMVKAM6FX8Z87 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.916" pid=897 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc gitlab # [ 356.130895] gitaly[897]: time="2026-09-14T08:40:39.931Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NNJZWCCMVKAM6FX8Z87 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.916" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 356.156672] gitlab-workhorse[820]: time="2026-09-14T08:40:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 356.160441] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:40:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 72 gitlab # [ 356.163288] gitaly[897]: time="2026-09-14T08:40:39.931Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6NNJZWCCMVKAM6FX8Z87 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:40:39.916" grpc.time_ms=15.15 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 368.926063] sidekiq-cluster[3414]: 2026-09-14T08:40:52.752Z pid=3414 tid=yri INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3414, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426086400} gitlab # [ 369.192234] sidekiq-cluster[3414]: W, [2026-09-14T08:40:53.020305 #3414] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 369.330560] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:40:53.158Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 372.313725] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:40:56.140Z","message":"GitLab reliable fetch activated!"} gitlab # [ 372.314205] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:40:56.141Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 408.282804] postgres[4249]: [4249] ERROR: relation "table_without_models" does not exist gitlab # [ 408.283426] postgres[4249]: [4249] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 413.042679] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:36.870Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 413.045504] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:36.873Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 413.047472] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:36.873Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 413.049643] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:36.873Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 413.052057] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:36.880Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 72.13 seconds) gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 413.439508] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:37 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 129 gitlab # [ 413.668312] gitaly[897]: time="2026-09-14T08:41:37.495Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8DQK65D3HM8J6TZWD5A7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:37.475" grpc.time_ms=19.302 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 413.707317] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:37 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 228 gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 68.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 414.166652] gitaly[897]: time="2026-09-14T08:41:37.994Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-09-14T08:42:07.080" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-09-14T08:41:37.980" grpc.time_ms=13.845 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 414.208336] gitaly[897]: time="2026-09-14T08:41:38.026Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK duration_ms=186ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 414.243189] gitaly[897]: time="2026-09-14T08:41:38.071Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK duration_ms=44ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 414.256373] gitaly[897]: time="2026-09-14T08:41:38.071Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5316 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:37.838" grpc.time_ms=232.983 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 414.344092] gitaly[897]: time="2026-09-14T08:41:38.172Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4786 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:38.170" grpc.time_ms=1.645 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 414.437383] gitaly[897]: time="2026-09-14T08:41:38.265Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4785 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:38.263" grpc.time_ms=1.81 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 414.527690] gitaly[897]: time="2026-09-14T08:41:38.355Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4787 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:38.353" grpc.time_ms=1.742 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 414.745025] gitaly[897]: time="2026-09-14T08:41:38.572Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK duration_ms=140ns method=POST pid=897 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 414.759645] gitaly[897]: time="2026-09-14T08:41:38.573Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4810 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:38.430" grpc.time_ms=142.626 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 414.791210] gitaly[897]: time="2026-09-14T08:41:38.595Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK diskcache=5e0dc136-34fa-4461-ac43-7e22c10e7af5 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:37.724" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # 28fc4ed..2974806 master -> master[ 414.816468] gitlab-workhorse[820]: time="2026-09-14T08:41:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 414.820423] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:38 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1029 gitlab # gitlab # [ 414.823800] gitaly[897]: time="2026-09-14T08:41:38.595Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=472 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13464 command.oublock=32 command.real_time_ms=862 command.spawn_token_fork_ms=2 command.system_time_ms=217 command.user_time_ms=255 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:37.724" grpc.time_ms=870.767 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.63 seconds) gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 415.016488] nixbot[3543]: INFO: ::ffff:127.0.0.1:43048 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.15 seconds) gitlab # [ 415.788578] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:39.614Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"40f4b39882ce2d67d77096fb","created_at":"2026-09-14T08:40:09.500Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:09.554Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-40f4b39882ce2d67d77096fb: start","job_status":"start","queue_duration_s":90.06062,"scheduling_latency_s":90.060719} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.05 seconds) gitlab # [ 416.071070] nixbot[3543]: INFO: ::ffff:127.0.0.1:43052 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab # [ 416.162923] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:39.990Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"40f4b39882ce2d67d77096fb","created_at":"2026-09-14T08:40:09.500Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:09.554Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-40f4b39882ce2d67d77096fb: done: 0.37576 sec","job_status":"done","queue_duration_s":90.06062,"scheduling_latency_s":90.060719,"redis_calls":23,"redis_duration_s":0.010214,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.005415,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002795,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002004,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.191046,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19475602,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.37576,"completed_at":"2026-09-14T08:41:39.990Z","load_balancing_strategy":"primary","db_duration_s":0.106104,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 416.265200] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:39.993Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ebecf5cfc3e9484d6c680f02","created_at":"2026-09-14T08:40:09.573Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:09.581Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ebecf5cfc3e9484d6c680f02: start","job_status":"start","queue_duration_s":90.411779,"scheduling_latency_s":90.411811} gitlab # [ 416.336831] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.164Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"ebecf5cfc3e9484d6c680f02","created_at":"2026-09-14T08:40:09.573Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:09.581Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-ebecf5cfc3e9484d6c680f02: done: 0.171234 sec","job_status":"done","queue_duration_s":90.411779,"scheduling_latency_s":90.411811,"redis_calls":14,"redis_duration_s":0.004939000000000001,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001057,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002279,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001603,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.124621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.13360054,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.171234,"completed_at":"2026-09-14T08:41:40.164Z","load_balancing_strategy":"primary","db_duration_s":0.036603,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 416.432640] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.166Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cff312711a8c98a5b38f15e4","created_at":"2026-09-14T08:40:10.415Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.428Z","job_size_bytes":5,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cff312711a8c98a5b38f15e4: start","job_status":"start","queue_duration_s":89.738561,"scheduling_latency_s":89.738588} gitlab # [ 416.556789] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.384Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"cff312711a8c98a5b38f15e4","created_at":"2026-09-14T08:40:10.415Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.428Z","job_size_bytes":5,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-cff312711a8c98a5b38f15e4: done: 0.217784 sec","job_status":"done","queue_duration_s":89.738561,"scheduling_latency_s":89.738588,"redis_calls":12,"redis_duration_s":0.003637,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00105,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002026,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000561,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169029,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045982,"duration_s":0.217784,"completed_at":"2026-09-14T08:41:40.384Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.041908,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 416.661492] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.386Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"58426f76126d738667b4c1ff","created_at":"2026-09-14T08:40:10.477Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.486Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-58426f76126d738667b4c1ff: start","job_status":"start","queue_duration_s":89.900417,"scheduling_latency_s":89.900449} gitlab # [ 416.828740] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.655Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"58426f76126d738667b4c1ff","created_at":"2026-09-14T08:40:10.477Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.486Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-58426f76126d738667b4c1ff: done: 0.268857 sec","job_status":"done","queue_duration_s":89.900417,"scheduling_latency_s":89.900449,"redis_calls":11,"redis_duration_s":0.003316,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.00079,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000634,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.042,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.227,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.227,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.04,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.166259,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00059932,"duration_s":0.268857,"completed_at":"2026-09-14T08:41:40.655Z","load_balancing_strategy":"primary","db_duration_s":0.10581,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.924294] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.657Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"15ad9dfe9f7aaec4e2867e59","created_at":"2026-09-14T08:40:10.492Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.500Z","job_size_bytes":89,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-15ad9dfe9f7aaec4e2867e59: start","job_status":"start","queue_duration_s":90.157074,"scheduling_latency_s":90.157096} gitlab # [ 416.944835] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.707Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"15ad9dfe9f7aaec4e2867e59","created_at":"2026-09-14T08:40:10.492Z","trace_propagation_headers":{"sentry-trace":"dd6539098f7843679a989bfee4d595fc-dcdde7c7b9ed47a0","baggage":"sentry-trace_id=dd6539098f7843679a989bfee4d595fc,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH5PC2G8YM43N0HARYX1GR","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:10.500Z","job_size_bytes":89,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-15ad9dfe9f7aaec4e2867e59: done: 0.050614 sec","job_status":"done","queue_duration_s":90.157074,"scheduling_latency_s":90.157096,"redis_calls":11,"redis_duration_s":0.003975,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000899,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001996,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00108,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00051516,"duration_s":0.050614,"completed_at":"2026-09-14T08:41:40.707Z","load_balancing_strategy":"primary","db_duration_s":0.009261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 416.989861] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.710Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 28fc4ed78983aa2a5da210e428415bbec71b479a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97daeb242aacab82ad46efc7","created_at":"2026-09-14T08:40:19.220Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-709d30e200bc42ca","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:40:19.230Z","job_size_bytes":185,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97daeb242aacab82ad46efc7: start","job_status":"start","queue_duration_s":81.478995,"scheduling_latency_s":81.479016} gitlab # [ 417.007672] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:40.730Z","message":"changes: \"0000000000000000000000000000000000000000 28fc4ed78983aa2a5da210e428415bbec71b479a refs/heads/master\\n\""} gitlab # [ 417.010554] gitaly[897]: time="2026-09-14T08:41:40.772Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:50.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:40.763" grpc.time_ms=9.066 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.023416] gitaly[897]: time="2026-09-14T08:41:40.799Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:41:50.023" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:40.783" grpc.time_ms=16.425 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 417.089821] gitaly[897]: time="2026-09-14T08:41:40.918Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-14T08:41:50.139" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:40.899" grpc.time_ms=17.953 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.150557] nixbot[3543]: INFO: ::ffff:127.0.0.1:43068 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.08 seconds) gitlab # [ 417.179922] gitaly[897]: time="2026-09-14T08:41:41.008Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:41.006" grpc.time_ms=0.999 pid=897 span.kind=server system=grpc gitlab # [ 417.233398] gitaly[897]: time="2026-09-14T08:41:41.061Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3939 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:41.060" grpc.time_ms=0.798 pid=897 span.kind=server system=grpc gitlab # [ 417.253560] gitaly[897]: time="2026-09-14T08:41:41.076Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 diskcache=d3dcc4cf-b9b1-4971-97ec-b47dd1bcba51 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:41:50.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:40.943" pid=897 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 417.271264] gitaly[897]: time="2026-09-14T08:41:41.077Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=136 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5092 command.oublock=8 command.real_time_ms=125 command.spawn_token_fork_ms=1 command.system_time_ms=74 command.user_time_ms=62 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:41:50.182" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:40.943" grpc.time_ms=133.763 pid=897 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 417.290289] gitaly[897]: time="2026-09-14T08:41:41.110Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-09-14T08:41:51.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:41.110" grpc.time_ms=0.376 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.316194] gitaly[897]: time="2026-09-14T08:41:41.144Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=860 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=21 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-09-14T08:41:51.017" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:41.118" grpc.time_ms=25.925 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.489324] gitaly[897]: time="2026-09-14T08:41:41.317Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-09-14T08:42:11.004" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:41.304" grpc.time_ms=12.395 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.548052] gitaly[897]: time="2026-09-14T08:41:41.376Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T08:41:51.274" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:41.375" grpc.time_ms=0.677 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.578537] gitaly[897]: time="2026-09-14T08:41:41.393Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-14T08:41:51.280" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-09-14T08:41:41.381" grpc.time_ms=11.938 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.610828] gitaly[897]: time="2026-09-14T08:41:41.439Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:51.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:41.421" grpc.time_ms=17.322 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.647349] gitaly[897]: time="2026-09-14T08:41:41.464Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:51.026" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:41.446" grpc.time_ms=17.654 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 417.689789] gitaly[897]: time="2026-09-14T08:41:41.518Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=24 command.spawn_token_fork_ms=3 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T08:42:11.188" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:41.489" grpc.time_ms=28.651 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 418.209344] nixbot[3543]: INFO: ::ffff:127.0.0.1:43076 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.06 seconds) gitlab # [ 418.259131] gitaly[897]: time="2026-09-14T08:41:42.086Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-14T08:41:52.003" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:42.083" grpc.time_ms=2.15 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 418.328938] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.154Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 28fc4ed78983aa2a5da210e428415bbec71b479a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"97daeb242aacab82ad46efc7","created_at":"2026-09-14T08:40:19.220Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-709d30e200bc42ca","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:40:19.230Z","job_size_bytes":185,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-97daeb242aacab82ad46efc7: done: 1.444886 sec","job_status":"done","queue_duration_s":81.478995,"scheduling_latency_s":81.479016,"gitaly_calls":13,"gitaly_duration_s":0.357827,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.06749,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.006484,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008963,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002364,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005175,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.030039,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.014465,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.088,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.303,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.349,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.088,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.945335,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00076314,"exclusive_lock_hold_duration_s":0.03367372,"duration_s":1.444886,"completed_at":"2026-09-14T08:41:42.154Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.154249,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 418.427141] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.158Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"03708735e49c484e0e8a01e3","created_at":"2026-09-14T08:40:26.861Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:26.879Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-03708735e49c484e0e8a01e3: start","job_status":"start","queue_duration_s":75.278789,"scheduling_latency_s":75.278827} gitlab # [ 418.480464] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.307Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"03708735e49c484e0e8a01e3","created_at":"2026-09-14T08:40:26.861Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:26.879Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-03708735e49c484e0e8a01e3: done: 0.149716 sec","job_status":"done","queue_duration_s":75.278789,"scheduling_latency_s":75.278827,"redis_calls":16,"redis_duration_s":0.005864,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003047,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000728,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.100205,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005423,"duration_s":0.149716,"completed_at":"2026-09-14T08:41:42.307Z","load_balancing_strategy":"primary","db_duration_s":0.037817,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 418.568653] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.310Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0d2bbe603217d7616ce39b21","created_at":"2026-09-14T08:40:26.880Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:26.892Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0d2bbe603217d7616ce39b21: start","job_status":"start","queue_duration_s":75.418172,"scheduling_latency_s":75.418202} gitlab # [ 418.662261] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.489Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"0d2bbe603217d7616ce39b21","created_at":"2026-09-14T08:40:26.880Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:26.892Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-0d2bbe603217d7616ce39b21: done: 0.178788 sec","job_status":"done","queue_duration_s":75.418172,"scheduling_latency_s":75.418202,"redis_calls":21,"redis_duration_s":0.008413,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003682,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002464,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002267,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.012,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.110313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00108562,"duration_s":0.178788,"completed_at":"2026-09-14T08:41:42.489Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.053286,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 418.756376] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:42.493Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"066c3244158ec1a1c94a1894","created_at":"2026-09-14T08:40:27.109Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:27.117Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-066c3244158ec1a1c94a1894: start","job_status":"start","queue_duration_s":75.375705,"scheduling_latency_s":75.375747} gitlab # [ 419.133543] gitaly[897]: time="2026-09-14T08:41:42.961Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:42.960" grpc.time_ms=0.801 pid=897 span.kind=server system=grpc gitlab # [ 419.185971] gitaly[897]: time="2026-09-14T08:41:43.014Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:43.013" grpc.time_ms=0.741 pid=897 span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 419.240810] gitaly[897]: time="2026-09-14T08:41:43.069Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:43.066" grpc.time_ms=1.038 pid=897 span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.06 seconds) gitlab # [ 419.270039] gitaly[897]: time="2026-09-14T08:41:43.083Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB diskcache=1f6227ec-236a-4e38-b89b-f22e43ec1e36 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:12.605" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:42.905" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.295125] gitaly[897]: time="2026-09-14T08:41:43.083Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=195 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7432 command.oublock=0 command.real_time_ms=170 command.spawn_token_fork_ms=1 command.system_time_ms=139 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:12.605" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:42.905" grpc.time_ms=177.525 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.320492] nixbot[3543]: INFO: ::ffff:127.0.0.1:43084 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab # [ 419.473353] gitaly[897]: time="2026-09-14T08:41:43.301Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:13.099" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:43.299" grpc.time_ms=1.576 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.559398] gitaly[897]: time="2026-09-14T08:41:43.387Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH689T531K88YQ1JZA2CYB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:13.185" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:43.386" grpc.time_ms=1.11 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.602054] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.428Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"066c3244158ec1a1c94a1894","created_at":"2026-09-14T08:40:27.109Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-27bfc8aa46084062","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:40:27.117Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-066c3244158ec1a1c94a1894: done: 0.934851 sec","job_status":"done","queue_duration_s":75.375705,"scheduling_latency_s":75.375747,"gitaly_calls":3,"gitaly_duration_s":0.191746,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.04973,"redis_read_bytes":455,"redis_write_bytes":27997,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.002308,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002442,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.014615,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.005235,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14491,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.017301,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000751,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000824,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003946,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.107,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.041,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.047,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.056,"db_ci_duration_s":0.051,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.596712,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.80435216,"duration_s":0.934851,"completed_at":"2026-09-14T08:41:43.428Z","load_balancing_strategy":"primary","db_duration_s":0.125873,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 419.690947] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.432Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","28fc4ed78983aa2a5da210e428415bbec71b479a 297480606dca023f7b934bf3a4c0ee99c9bd7e3a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"466a10ba1300a1b2d7baad25","created_at":"2026-09-14T08:41:38.518Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-f43c378cb34c417b","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:38.524Z","job_size_bytes":185,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-466a10ba1300a1b2d7baad25: start","job_status":"start","queue_duration_s":4.907916,"scheduling_latency_s":4.907956} gitlab # [ 419.711251] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.455Z","message":"changes: \"28fc4ed78983aa2a5da210e428415bbec71b479a 297480606dca023f7b934bf3a4c0ee99c9bd7e3a refs/heads/master\\n\""} gitlab # [ 419.714339] gitaly[897]: time="2026-09-14T08:41:43.504Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=7 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:53.067" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:43.487" grpc.time_ms=17.06 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.729981] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.535Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"e344dfae59b1f401a8703243","created_at":"2026-09-14T08:41:43.532Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"94e4b3143e6d25823d201c70","job_size_bytes":46,"pid":3414,"job_status":"deduplicated","message":"ProjectCacheWorker JID-e344dfae59b1f401a8703243: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 419.745527] gitaly[897]: time="2026-09-14T08:41:43.528Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-09-14T08:41:53.093" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:43.514" grpc.time_ms=13.671 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.758896] gitaly[897]: time="2026-09-14T08:41:43.579Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-09-14T08:41:53.141" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:43.561" grpc.time_ms=16.971 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.778684] gitaly[897]: time="2026-09-14T08:41:43.606Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-09-14T08:42:13.387" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:43.587" grpc.time_ms=18.938 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.809671] gitaly[897]: time="2026-09-14T08:41:43.637Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-09-14T08:41:53.208" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-09-14T08:41:43.628" grpc.time_ms=9.431 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 419.840654] gitaly[897]: time="2026-09-14T08:41:43.668Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:53.233" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:43.653" grpc.time_ms=15.424 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.020390] gitaly[897]: time="2026-09-14T08:41:43.848Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-09-14T08:41:53.005" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:43.846" grpc.time_ms=2.184 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.059491] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.881Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","28fc4ed78983aa2a5da210e428415bbec71b479a 297480606dca023f7b934bf3a4c0ee99c9bd7e3a refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"466a10ba1300a1b2d7baad25","created_at":"2026-09-14T08:41:38.518Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-f43c378cb34c417b","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:38.524Z","job_size_bytes":185,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-466a10ba1300a1b2d7baad25: done: 0.448906 sec","job_status":"done","queue_duration_s":4.907916,"scheduling_latency_s":4.907956,"gitaly_calls":7,"gitaly_duration_s":0.121357,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.0365,"redis_read_bytes":501,"redis_write_bytes":11943,"redis_cache_calls":9,"redis_cache_duration_s":0.003929,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001604,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5720,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005823,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.01167,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.013474,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.047,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.047,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.282313,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00105164,"exclusive_lock_hold_duration_s":0.03381042,"duration_s":0.448906,"completed_at":"2026-09-14T08:41:43.881Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043721,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 420.152209] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:43.883Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"94e4b3143e6d25823d201c70","created_at":"2026-09-14T08:41:40.804Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:40.845Z","job_size_bytes":46,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-94e4b3143e6d25823d201c70: start","job_status":"start","queue_duration_s":3.038262,"scheduling_latency_s":3.038293} gitlab # [ 420.172756] gitaly[897]: time="2026-09-14T08:41:43.965Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-09-14T14:41:43.954" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:43.955" grpc.time_ms=9.922 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.185339] gitaly[897]: time="2026-09-14T08:41:43.994Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-14T08:42:13.776" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:43.976" grpc.time_ms=17.445 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.205831] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.033Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"94e4b3143e6d25823d201c70","created_at":"2026-09-14T08:41:40.804Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:40.845Z","job_size_bytes":46,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-94e4b3143e6d25823d201c70: done: 0.150117 sec","job_status":"done","queue_duration_s":3.038262,"scheduling_latency_s":3.038293,"gitaly_calls":2,"gitaly_duration_s":0.03388,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.011103,"redis_read_bytes":69,"redis_write_bytes":4882,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001937,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000798,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2115,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003439,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.003555,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":748,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001374,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040518,"duration_s":0.150117,"completed_at":"2026-09-14T08:41:44.033Z","load_balancing_strategy":"primary","db_duration_s":0.019059,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 420.259347] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.035Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"78f08f734770683431ac2de5","created_at":"2026-09-14T08:41:40.880Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:40.891Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-78f08f734770683431ac2de5: start","job_status":"start","queue_duration_s":3.143835,"scheduling_latency_s":3.143859} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 420.283210] gitaly[897]: time="2026-09-14T08:41:44.111Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 diskcache=a54d36be-5533-4c90-82f7-33ba5d6cc750 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T14:41:44.073" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.074" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.292550] gitaly[897]: time="2026-09-14T08:41:44.111Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=26 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1174 command.oublock=0 command.real_time_ms=46 command.spawn_token_fork_ms=4 command.system_time_ms=26 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T14:41:44.073" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.074" grpc.time_ms=37.274 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.354323] nixbot[3543]: INFO: ::ffff:127.0.0.1:43088 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.08 seconds) gitlab # [ 420.367996] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.195Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"78f08f734770683431ac2de5","created_at":"2026-09-14T08:41:40.880Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:40.891Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-78f08f734770683431ac2de5: done: 0.159696 sec","job_status":"done","queue_duration_s":3.143835,"scheduling_latency_s":3.143859,"gitaly_calls":1,"gitaly_duration_s":0.041212,"redis_calls":15,"redis_duration_s":0.005661,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000513,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002268,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001057,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001823,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.021,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.041,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.088527,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11874988,"duration_s":0.159696,"completed_at":"2026-09-14T08:41:44.195Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022064,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 420.429101] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.198Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"1d583619015b50123f2cd5f0","created_at":"2026-09-14T08:41:41.101Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:41.105Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1d583619015b50123f2cd5f0: start","job_status":"start","queue_duration_s":3.093144,"scheduling_latency_s":3.093199} gitlab # [ 420.444672] gitaly[897]: time="2026-09-14T08:41:44.264Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-09-14T08:42:14.047" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.248" grpc.time_ms=16.401 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.458805] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.286Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"1d583619015b50123f2cd5f0","created_at":"2026-09-14T08:41:41.101Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:41.105Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-1d583619015b50123f2cd5f0: done: 0.087559 sec","job_status":"done","queue_duration_s":3.093144,"scheduling_latency_s":3.093199,"gitaly_calls":1,"gitaly_duration_s":0.019408,"redis_calls":16,"redis_duration_s":0.005933,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000496,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001775,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.002447,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05456,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.087559,"completed_at":"2026-09-14T08:41:44.285Z","load_balancing_strategy":"primary","db_duration_s":0.005193,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 420.503331] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:44.289Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"661dc011cce09496b3c8ae2f","created_at":"2026-09-14T08:41:41.894Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d23199e75f0a3416847f28ee4e8b8a42e890316589a4e3c54b2810102d95a063","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:41.900Z","job_size_bytes":467,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-661dc011cce09496b3c8ae2f: start","job_status":"start","queue_duration_s":2.388327,"scheduling_latency_s":2.388365} gitlab # [ 420.520456] gitaly[897]: time="2026-09-14T08:41:44.342Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:14.140" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.341" grpc.time_ms=0.879 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.534796] gitaly[897]: time="2026-09-14T08:41:44.362Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T08:41:54.265" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:44.346" grpc.time_ms=16.246 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.744721] gitaly[897]: time="2026-09-14T08:41:44.572Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:14.371" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.571" grpc.time_ms=1.332 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.780513] gitaly[897]: time="2026-09-14T08:41:44.581Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=29 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-14T08:42:14.376" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:44.577" grpc.time_ms=3.664 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 420.870810] gitaly[897]: time="2026-09-14T08:41:44.698Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:54.261" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:44.682" grpc.time_ms=16.555 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 421.417961] nixbot[3543]: INFO: ::ffff:127.0.0.1:59116 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.07 seconds) gitlab # [ 421.653275] gitaly[897]: time="2026-09-14T08:41:45.480Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=13 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T08:42:15.253" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:45.454" grpc.time_ms=25.891 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.694477] gitaly[897]: time="2026-09-14T08:41:45.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=12 component=gitaly.StreamServerInterceptor correlation_id=01M2FH6021RCZJYXE1RH1W2DG4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-14T08:42:15.283" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:45.483" grpc.time_ms=23.735 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 421.724613] sidekiq-cluster[3414]: {"severity":"WARN","time":"2026-09-14T08:41:45.547Z","class":"CreatePipelineWorker","correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"28fc4ed78983aa2a5da210e428415bbec71b479a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"28fc4ed78983aa2a5da210e428415bbec71b479a","ref":"refs/heads/master"},"retry":0} gitlab # [ 421.737041] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"661dc011cce09496b3c8ae2f","created_at":"2026-09-14T08:41:41.894Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d23199e75f0a3416847f28ee4e8b8a42e890316589a4e3c54b2810102d95a063","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:41.900Z","job_size_bytes":467,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-661dc011cce09496b3c8ae2f: done: 1.268253 sec","job_status":"done","queue_duration_s":2.388327,"scheduling_latency_s":2.388365,"gitaly_calls":7,"gitaly_duration_s":0.108788,"redis_calls":51,"redis_duration_s":0.015249,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001166,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.007214,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003666,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.044,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.08061,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040916,"duration_s":1.268253,"completed_at":"2026-09-14T08:41:45.557Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.078486,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 421.804815] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.559Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"1f5a0a30-fa6a-44d2-a139-ee4552053717\"}"],"class":"WebHookWorker","jid":"90165ced96cbaaeb0f26d2ce","created_at":"2026-09-14T08:41:42.046Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9882eedb35e409db44434d237be3dd4d656e4f4e06d90aeb79e0ddb226b2954d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:42.052Z","job_size_bytes":1751,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90165ced96cbaaeb0f26d2ce: start","job_status":"start","queue_duration_s":3.507391,"scheduling_latency_s":3.507417} gitlab # [ 421.920364] nixbot[3543]: INFO: ::ffff:127.0.0.1:59118 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 421.956455] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.783Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"1f5a0a30-fa6a-44d2-a139-ee4552053717\"}"],"class":"WebHookWorker","jid":"90165ced96cbaaeb0f26d2ce","created_at":"2026-09-14T08:41:42.046Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-91db57c836bf49e5","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9882eedb35e409db44434d237be3dd4d656e4f4e06d90aeb79e0ddb226b2954d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:42.052Z","job_size_bytes":1751,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90165ced96cbaaeb0f26d2ce: done: 0.223409 sec","job_status":"done","queue_duration_s":3.507391,"scheduling_latency_s":3.507417,"redis_calls":19,"redis_duration_s":0.006317,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001522,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000848,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002641,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001306,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09631166000002622,"cpu_s":0.108007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00066052,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.223409,"completed_at":"2026-09-14T08:41:45.783Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004645,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.051234] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.787Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"81dc3e37a4343ae457757c97","created_at":"2026-09-14T08:41:42.883Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:42.890Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-81dc3e37a4343ae457757c97: start","job_status":"start","queue_duration_s":2.896476,"scheduling_latency_s":2.896515} gitlab # [ 422.075145] gitaly[897]: time="2026-09-14T08:41:45.903Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8P268TTGVMH6TB8X4TXY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:45.884" grpc.time_ms=18.746 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 422.114234] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 107 gitlab # [ 422.119150] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.906Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"81dc3e37a4343ae457757c97","created_at":"2026-09-14T08:41:42.883Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:42.890Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-81dc3e37a4343ae457757c97: done: 0.119781 sec","job_status":"done","queue_duration_s":2.896476,"scheduling_latency_s":2.896515,"redis_calls":10,"redis_duration_s":0.005142,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000547,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004007,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000588,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032814,"duration_s":0.119781,"completed_at":"2026-09-14T08:41:45.906Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021466,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 422.184163] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.909Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8d4d97ae17d619ab9edbbbf4","created_at":"2026-09-14T08:41:43.100Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.118Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8d4d97ae17d619ab9edbbbf4: start","job_status":"start","queue_duration_s":2.790936,"scheduling_latency_s":2.79096} gitlab # [ 422.203857] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.955Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8d4d97ae17d619ab9edbbbf4","created_at":"2026-09-14T08:41:43.100Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.118Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8d4d97ae17d619ab9edbbbf4: done: 0.045685 sec","job_status":"done","queue_duration_s":2.790936,"scheduling_latency_s":2.79096,"redis_calls":10,"redis_duration_s":0.0018870000000000002,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000369,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001069,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03057,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002924,"duration_s":0.045685,"completed_at":"2026-09-14T08:41:45.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005788,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.251988] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.956Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9983737d474b45762b47de56","created_at":"2026-09-14T08:41:43.146Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.159Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9983737d474b45762b47de56: start","job_status":"start","queue_duration_s":2.797055,"scheduling_latency_s":2.797073} gitlab # [ 422.267903] gitaly[897]: time="2026-09-14T08:41:46.001Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P5R376FHH0ZSX9GSJS9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:45.986" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 422.278583] gitlab-workhorse[820]: time="2026-09-14T08:41:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 422.280817] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 90 gitlab # [ 422.282625] gitlab-workhorse[820]: time="2026-09-14T08:41:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=338 gitlab # [ 422.284455] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.55.0" 87 gitlab # [ 422.286311] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.995Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"9983737d474b45762b47de56","created_at":"2026-09-14T08:41:43.146Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.159Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-9983737d474b45762b47de56: done: 0.038456 sec","job_status":"done","queue_duration_s":2.797055,"scheduling_latency_s":2.797073,"redis_calls":10,"redis_duration_s":0.001695,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001014,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000372,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026078,"duration_s":0.038456,"completed_at":"2026-09-14T08:41:45.995Z","load_balancing_strategy":"primary","db_duration_s":0.005263,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.329630] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:45.996Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fa9b30ec8296db61b825c7b1","created_at":"2026-09-14T08:41:43.187Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.196Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fa9b30ec8296db61b825c7b1: start","job_status":"start","queue_duration_s":2.800057,"scheduling_latency_s":2.800072} gitlab # [ 422.345050] gitaly[897]: time="2026-09-14T08:41:46.002Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P5R376FHH0ZSX9GSJS9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:45.986" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 422.358228] nixbot[3543]: {"ts": "2026-09-14T08:41:46.145Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 422.360615] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.038Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fa9b30ec8296db61b825c7b1","created_at":"2026-09-14T08:41:43.187Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.196Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fa9b30ec8296db61b825c7b1: done: 0.042356 sec","job_status":"done","queue_duration_s":2.800057,"scheduling_latency_s":2.800072,"redis_calls":13,"redis_duration_s":0.004187,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000384,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002892,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000403,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027064,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030616,"duration_s":0.042356,"completed_at":"2026-09-14T08:41:46.038Z","load_balancing_strategy":"primary","db_duration_s":0.006002,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.405541] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.040Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"581c24831dbef00a353845b1","created_at":"2026-09-14T08:41:43.197Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.205Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-581c24831dbef00a353845b1: start","job_status":"start","queue_duration_s":2.835002,"scheduling_latency_s":2.835019} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 422.421669] gitaly[897]: time="2026-09-14T08:41:46.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P5R376FHH0ZSX9GSJS9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:45.986" grpc.time_ms=16.155 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.05 seconds) gitlab # [ 422.435027] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.075Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"581c24831dbef00a353845b1","created_at":"2026-09-14T08:41:43.197Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.205Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-581c24831dbef00a353845b1: done: 0.035525 sec","job_status":"done","queue_duration_s":2.835002,"scheduling_latency_s":2.835019,"redis_calls":10,"redis_duration_s":0.0018059999999999999,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000329,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001168,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000309,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000208,"duration_s":0.035525,"completed_at":"2026-09-14T08:41:46.075Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004303,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.477660] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.077Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1e84d9b5563e6d3983303ca5","created_at":"2026-09-14T08:41:43.206Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.216Z","job_size_bytes":19,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1e84d9b5563e6d3983303ca5: start","job_status":"start","queue_duration_s":2.861061,"scheduling_latency_s":2.861078} gitlab # [ 422.493419] gitaly[897]: time="2026-09-14T08:41:46.092Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P8NRGK5ZFWNK9D4HWT4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.062" pid=897 remote_ip="2001:db8:1::1" request_sha=58534e1fcdc639f4492392bf3c11010af0a659b6 response_bytes=338 span.kind=server system=grpc gitlab # [ 422.503664] nixbot[3543]: INFO: ::ffff:127.0.0.1:59128 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab # [ 422.505683] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.109Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"1e84d9b5563e6d3983303ca5","created_at":"2026-09-14T08:41:43.206Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.216Z","job_size_bytes":19,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-1e84d9b5563e6d3983303ca5: done: 0.032389 sec","job_status":"done","queue_duration_s":2.861061,"scheduling_latency_s":2.861078,"redis_calls":11,"redis_duration_s":0.001741,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000359,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021314,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002122,"duration_s":0.032389,"completed_at":"2026-09-14T08:41:46.109Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004309,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.548647] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.111Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"37febf98722c488f97f20db4","created_at":"2026-09-14T08:41:43.221Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.228Z","job_size_bytes":66,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-37febf98722c488f97f20db4: start","job_status":"start","queue_duration_s":2.883189,"scheduling_latency_s":2.883207} gitlab # [ 422.565315] gitaly[897]: time="2026-09-14T08:41:46.092Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=4 command.system_time_ms=15 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P8NRGK5ZFWNK9D4HWT4 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.062" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 422.579670] nixbot[3543]: {"ts": "2026-09-14T08:41:46.350Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 422.582548] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.153Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"37febf98722c488f97f20db4","created_at":"2026-09-14T08:41:43.221Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.228Z","job_size_bytes":66,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-37febf98722c488f97f20db4: done: 0.041898 sec","job_status":"done","queue_duration_s":2.883189,"scheduling_latency_s":2.883207,"redis_calls":10,"redis_duration_s":0.002152,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000403,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001307,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036318,"duration_s":0.041898,"completed_at":"2026-09-14T08:41:46.153Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.626614] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.155Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"fa3433b63a38a07372c0c3f9","created_at":"2026-09-14T08:41:43.229Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.237Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-fa3433b63a38a07372c0c3f9: start","job_status":"start","queue_duration_s":2.917652,"scheduling_latency_s":2.917675} gitlab # [ 422.643131] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 185 gitlab # [ 422.645663] gitaly[897]: time="2026-09-14T08:41:46.093Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=27 command.spawn_token_fork_ms=4 command.system_time_ms=15 command.user_time_ms=9 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8P8NRGK5ZFWNK9D4HWT4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.062" grpc.time_ms=30.873 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 422.661044] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.224Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"fa3433b63a38a07372c0c3f9","created_at":"2026-09-14T08:41:43.229Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.237Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-fa3433b63a38a07372c0c3f9: done: 0.069747 sec","job_status":"done","queue_duration_s":2.917652,"scheduling_latency_s":2.917675,"redis_calls":10,"redis_duration_s":0.003656,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000716,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002428,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000512,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.05101,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00080384,"duration_s":0.069747,"completed_at":"2026-09-14T08:41:46.224Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007164,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.705484] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.228Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b16b7e35a34608b4bb8697e7","created_at":"2026-09-14T08:41:43.247Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.253Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b16b7e35a34608b4bb8697e7: start","job_status":"start","queue_duration_s":2.9744,"scheduling_latency_s":2.974447} gitlab # [ 422.722262] gitaly[897]: time="2026-09-14T08:41:46.217Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:16.016" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.216" grpc.time_ms=0.617 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.735131] nixbot[3543]: {"ts": "2026-09-14T08:41:46.512Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 422.738560] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.306Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"b16b7e35a34608b4bb8697e7","created_at":"2026-09-14T08:41:43.247Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.253Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-b16b7e35a34608b4bb8697e7: done: 0.078722 sec","job_status":"done","queue_duration_s":2.9744,"scheduling_latency_s":2.974447,"redis_calls":10,"redis_duration_s":0.002438,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000607,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047772,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026226,"duration_s":0.078722,"completed_at":"2026-09-14T08:41:46.306Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.783759] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.308Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"216d4b43ae682071478dc37d","created_at":"2026-09-14T08:41:43.254Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.262Z","job_size_bytes":6,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-216d4b43ae682071478dc37d: start","job_status":"start","queue_duration_s":3.046203,"scheduling_latency_s":3.04622} gitlab # [ 422.799538] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 152 gitlab # [ 422.802190] gitaly[897]: time="2026-09-14T08:41:46.291Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:56.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:46.276" grpc.time_ms=14.87 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.815924] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.332Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"216d4b43ae682071478dc37d","created_at":"2026-09-14T08:41:43.254Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.262Z","job_size_bytes":6,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-216d4b43ae682071478dc37d: done: 0.023956 sec","job_status":"done","queue_duration_s":3.046203,"scheduling_latency_s":3.04622,"redis_calls":10,"redis_duration_s":0.001942,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001292,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002443,"duration_s":0.023956,"completed_at":"2026-09-14T08:41:46.332Z","load_balancing_strategy":"primary","db_duration_s":0.002098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.859775] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2bebfd9f46cfcf966f1827cf","created_at":"2026-09-14T08:41:43.262Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.268Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2bebfd9f46cfcf966f1827cf: start","job_status":"start","queue_duration_s":3.066405,"scheduling_latency_s":3.066422} gitlab # [ 422.874723] gitaly[897]: time="2026-09-14T08:41:46.407Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PKPX5NWJKJ1FBEWT4TR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:16.205" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.406" grpc.time_ms=0.631 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.887276] nixbot[3543]: {"ts": "2026-09-14T08:41:46.656Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a \"HTTP/1.1 201 Created\""} gitlab # [ 422.890152] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.376Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"2bebfd9f46cfcf966f1827cf","created_at":"2026-09-14T08:41:43.262Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.268Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-2bebfd9f46cfcf966f1827cf: done: 0.041743 sec","job_status":"done","queue_duration_s":3.066405,"scheduling_latency_s":3.066422,"redis_calls":10,"redis_duration_s":0.001639,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000336,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000883,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00042,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025184,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002426,"duration_s":0.041743,"completed_at":"2026-09-14T08:41:46.376Z","load_balancing_strategy":"primary","db_duration_s":0.010862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 422.932278] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.378Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f6f9962d76632316156cecad","created_at":"2026-09-14T08:41:43.277Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.283Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f6f9962d76632316156cecad: start","job_status":"start","queue_duration_s":3.094613,"scheduling_latency_s":3.09463} gitlab # [ 422.948623] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/28fc4ed78983aa2a5da210e428415bbec71b479a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 138 gitlab # [ 422.951086] gitaly[897]: time="2026-09-14T08:41:46.452Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PKPX5NWJKJ1FBEWT4TR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:56.168" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:46.439" grpc.time_ms=13.256 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 422.964420] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.454Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"f6f9962d76632316156cecad","created_at":"2026-09-14T08:41:43.277Z","trace_propagation_headers":{"sentry-trace":"c9b23a4d117749d49d2b3fb3e26db463-40f646b3951c4a18","baggage":"sentry-trace_id=c9b23a4d117749d49d2b3fb3e26db463,sentry-release=Unknown"},"correlation_id":"01M2FH689T531K88YQ1JZA2CYB","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.283Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-f6f9962d76632316156cecad: done: 0.07632 sec","job_status":"done","queue_duration_s":3.094613,"scheduling_latency_s":3.09463,"redis_calls":14,"redis_duration_s":0.003025,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000285,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000818,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001922,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027432,"duration_s":0.07632,"completed_at":"2026-09-14T08:41:46.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.008745] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.459Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d8ed47f49558ebf9c1e513b9","created_at":"2026-09-14T08:41:43.549Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"78f08f734770683431ac2de5","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.554Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d8ed47f49558ebf9c1e513b9: start","job_status":"start","queue_duration_s":2.904665,"scheduling_latency_s":2.904685} gitlab # [ 423.025487] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 423.027539] gitaly[897]: time="2026-09-14T08:41:46.493Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK diskcache=14fa4a86-02c9-4306-b62f-0044314b4905 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T14:41:46.482" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.482" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.037533] gitaly[897]: time="2026-09-14T08:41:46.494Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-09-14T14:41:46.482" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.482" grpc.time_ms=11.179 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.050419] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.527Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"d8ed47f49558ebf9c1e513b9","created_at":"2026-09-14T08:41:43.549Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"78f08f734770683431ac2de5","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.554Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-d8ed47f49558ebf9c1e513b9: done: 0.068124 sec","job_status":"done","queue_duration_s":2.904665,"scheduling_latency_s":2.904685,"gitaly_calls":1,"gitaly_duration_s":0.014347,"redis_calls":12,"redis_duration_s":0.004116,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001195,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.03990088,"duration_s":0.068124,"completed_at":"2026-09-14T08:41:46.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008462,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.095295] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:46.530Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5f6150314ef2a690d684cf73","created_at":"2026-09-14T08:41:43.778Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc42f617958f14d3bc27a7d66aa288699e00da067e31167113219f6a23877b4a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.783Z","job_size_bytes":467,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5f6150314ef2a690d684cf73: start","job_status":"start","queue_duration_s":2.746564,"scheduling_latency_s":2.746599} gitlab # [ 423.112343] gitlab-workhorse[820]: time="2026-09-14T08:41:46Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 423.114576] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:46 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 79 gitlab # [ 423.116492] gitaly[897]: time="2026-09-14T08:41:46.563Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PRM05NAWY6NP42NM6HG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:16.362" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.562" grpc.time_ms=0.909 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.129133] gitaly[897]: time="2026-09-14T08:41:46.571Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:16.368" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.569" grpc.time_ms=0.963 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.141257] gitaly[897]: time="2026-09-14T08:41:46.590Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T08:41:56.154" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:46.575" grpc.time_ms=14.328 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.154074] gitaly[897]: time="2026-09-14T08:41:46.607Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PRM05NAWY6NP42NM6HG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:56.226" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:46.597" grpc.time_ms=10.462 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.167447] gitaly[897]: time="2026-09-14T08:41:46.656Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:16.449" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.650" grpc.time_ms=2.597 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.180109] gitaly[897]: time="2026-09-14T08:41:46.665Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=237 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-09-14T08:42:16.460" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:46.660" grpc.time_ms=4.271 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.196753] gitaly[897]: time="2026-09-14T08:41:46.721Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:56.290" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:46.710" grpc.time_ms=10.332 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.209564] gitaly[897]: time="2026-09-14T08:41:46.777Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PY9D06WW5ZBSF0XPY3Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.766" grpc.time_ms=10.151 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 423.222298] gitaly[897]: time="2026-09-14T08:41:46.857Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8Q0VVRXE25YKWRNNX9CS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.846" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 423.232342] gitaly[897]: time="2026-09-14T08:41:46.857Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8Q0VVRXE25YKWRNNX9CS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.846" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 423.245361] gitaly[897]: time="2026-09-14T08:41:46.857Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8Q0VVRXE25YKWRNNX9CS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:46.846" grpc.time_ms=11.292 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.09 seconds) gitlab # [ 423.560336] nixbot[3543]: INFO: ::ffff:127.0.0.1:59142 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab # [ 423.604596] gitaly[897]: time="2026-09-14T08:41:47.432Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=3 command.system_time_ms=14 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-09-14T08:42:17.205" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:47.405" grpc.time_ms=26.918 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.646853] gitaly[897]: time="2026-09-14T08:41:47.457Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8E0Y8X1DF55H7GBZNFDK grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-09-14T08:42:17.235" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:47.436" grpc.time_ms=21.307 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 423.675038] sidekiq-cluster[3414]: {"severity":"WARN","time":"2026-09-14T08:41:47.474Z","class":"CreatePipelineWorker","correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"28fc4ed78983aa2a5da210e428415bbec71b479a","after":"297480606dca023f7b934bf3a4c0ee99c9bd7e3a","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"297480606dca023f7b934bf3a4c0ee99c9bd7e3a","ref":"refs/heads/master"},"retry":0} gitlab # [ 423.688257] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.481Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"5f6150314ef2a690d684cf73","created_at":"2026-09-14T08:41:43.778Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc42f617958f14d3bc27a7d66aa288699e00da067e31167113219f6a23877b4a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.783Z","job_size_bytes":467,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-5f6150314ef2a690d684cf73: done: 0.951565 sec","job_status":"done","queue_duration_s":2.746564,"scheduling_latency_s":2.746599,"gitaly_calls":7,"gitaly_duration_s":0.107796,"redis_calls":33,"redis_duration_s":0.00793,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.000983,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0015,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003312,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002135,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.822117,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.951565,"completed_at":"2026-09-14T08:41:47.481Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030306,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.749466] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.485Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"98060aa4-5bdc-49b3-92ce-91f392a72449\"}"],"class":"WebHookWorker","jid":"90232a998b69fab0976b6060","created_at":"2026-09-14T08:41:43.815Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8133a1861e247f9525458144d8b147bac063950fb3fb53878572ee62dfe46234","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.820Z","job_size_bytes":1749,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90232a998b69fab0976b6060: start","job_status":"start","queue_duration_s":3.665116,"scheduling_latency_s":3.665168} gitlab # [ 423.766917] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.575Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"98060aa4-5bdc-49b3-92ce-91f392a72449\"}"],"class":"WebHookWorker","jid":"90232a998b69fab0976b6060","created_at":"2026-09-14T08:41:43.815Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-10b906ede8be4d98","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8133a1861e247f9525458144d8b147bac063950fb3fb53878572ee62dfe46234","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:43.820Z","job_size_bytes":1749,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-90232a998b69fab0976b6060: done: 0.09003 sec","job_status":"done","queue_duration_s":3.665116,"scheduling_latency_s":3.665168,"redis_calls":14,"redis_duration_s":0.007583,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.002239,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003308,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002036,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.011368500000003223,"cpu_s":0.058989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.09003,"completed_at":"2026-09-14T08:41:47.575Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004219,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.814333] nixbot[3543]: INFO: ::ffff:127.0.0.1:59150 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 423.815587] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.582Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c1ab4a65f8efe66613abcbb7","created_at":"2026-09-14T08:41:44.008Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-ee04e943bbcc4d46","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:44.015Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c1ab4a65f8efe66613abcbb7: start","job_status":"start","queue_duration_s":3.566869,"scheduling_latency_s":3.566922} gitlab # [ 423.845586] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"c1ab4a65f8efe66613abcbb7","created_at":"2026-09-14T08:41:44.008Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-ee04e943bbcc4d46","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:44.015Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-c1ab4a65f8efe66613abcbb7: done: 0.090849 sec","job_status":"done","queue_duration_s":3.566869,"scheduling_latency_s":3.566922,"redis_calls":19,"redis_duration_s":0.009192,"redis_read_bytes":21,"redis_write_bytes":4287,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001481,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000571,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2258,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.005975,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001165,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.061477,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.012042500000000001,"duration_s":0.090849,"completed_at":"2026-09-14T08:41:47.673Z","load_balancing_strategy":"primary","db_duration_s":0.013784,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.890001] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.676Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90165ced96cbaaeb0f26d2ce"],"class":"WebHooks::LogExecutionWorker","jid":"e6a921d7d7950cfa0c6cc3cf","created_at":"2026-09-14T08:41:45.754Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-c61b7116e2dc4752","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9240695cde01de0ad159599c9a8fac7077ed44acd3283a7480a6369d60221196","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:45.765Z","job_size_bytes":2593,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e6a921d7d7950cfa0c6cc3cf: start","job_status":"start","queue_duration_s":1.910409,"scheduling_latency_s":1.910437} gitlab # [ 423.906824] gitaly[897]: time="2026-09-14T08:41:47.703Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8QTCMZM0W0QPRZEF5DQ4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.685" grpc.time_ms=17.382 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 423.920034] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 106 gitlab # [ 423.922093] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.747Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90165ced96cbaaeb0f26d2ce"],"class":"WebHooks::LogExecutionWorker","jid":"e6a921d7d7950cfa0c6cc3cf","created_at":"2026-09-14T08:41:45.754Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-c61b7116e2dc4752","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:9240695cde01de0ad159599c9a8fac7077ed44acd3283a7480a6369d60221196","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:45.765Z","job_size_bytes":2593,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e6a921d7d7950cfa0c6cc3cf: done: 0.071308 sec","job_status":"done","queue_duration_s":1.910409,"scheduling_latency_s":1.910437,"redis_calls":10,"redis_duration_s":0.003773,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00098,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000657,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.01,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.01,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044813,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00042988,"duration_s":0.071308,"completed_at":"2026-09-14T08:41:47.747Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017436,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 423.965803] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:47.749Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbaddb98520d1056c5935ad3","created_at":"2026-09-14T08:41:46.329Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-070c5ac5740b4dba","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:46.332Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fbaddb98520d1056c5935ad3: start","job_status":"start","queue_duration_s":1.417018,"scheduling_latency_s":1.417042} gitlab # [ 423.981154] gitaly[897]: time="2026-09-14T08:41:47.782Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8QXSY113K5V965Q5GT8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.769" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 423.991253] gitlab-workhorse[820]: time="2026-09-14T08:41:47Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 423.993149] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:47 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 79 gitlab # [ 423.994977] gitaly[897]: time="2026-09-14T08:41:47.782Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8QXSY113K5V965Q5GT8W grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.769" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 424.009232] gitaly[897]: time="2026-09-14T08:41:47.784Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8QXSY113K5V965Q5GT8W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.769" grpc.time_ms=13.954 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 424.081533] gitaly[897]: time="2026-09-14T08:41:47.909Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:47.908" grpc.time_ms=0.539 pid=897 span.kind=server system=grpc gitlab # [ 424.101350] gitaly[897]: time="2026-09-14T08:41:47.928Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R28FXZ2DX00P3B7WGQD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=206 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.914" grpc.time_ms=14.194 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 424.122179] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:47 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 80 gitlab # [ 424.125321] gitaly[897]: time="2026-09-14T08:41:47.948Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:47.947" grpc.time_ms=0.437 pid=897 span.kind=server system=grpc gitlab # [ 424.152118] gitaly[897]: time="2026-09-14T08:41:47.980Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:47.979" grpc.time_ms=0.488 pid=897 span.kind=server system=grpc gitlab # [ 424.169894] gitaly[897]: time="2026-09-14T08:41:47.989Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W diskcache=dedd6300-ed0b-4050-9d0c-43e6390c3d67 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:17.658" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:47.859" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.186048] gitaly[897]: time="2026-09-14T08:41:47.989Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=122 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7290 command.oublock=0 command.real_time_ms=123 command.spawn_token_fork_ms=1 command.system_time_ms=59 command.user_time_ms=63 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:17.658" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:47.859" grpc.time_ms=130.416 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.206197] gitaly[897]: time="2026-09-14T08:41:48.003Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R4TYK4KX0ANGB9QY86M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.986" pid=897 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 424.220526] gitlab-workhorse[820]: time="2026-09-14T08:41:48Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 424.222866] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:48 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 73 gitlab # [ 424.225326] gitaly[897]: time="2026-09-14T08:41:48.003Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R4TYK4KX0ANGB9QY86M grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.986" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 424.241602] gitaly[897]: time="2026-09-14T08:41:48.003Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=9 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R4TYK4KX0ANGB9QY86M grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:47.986" grpc.time_ms=17.691 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 424.258346] gitaly[897]: time="2026-09-14T08:41:48.074Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:18.072" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.073" grpc.time_ms=0.877 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.274657] gitaly[897]: time="2026-09-14T08:41:48.103Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:18.001" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.102" grpc.time_ms=0.669 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.291670] gitaly[897]: time="2026-09-14T08:41:48.119Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-09-14T08:42:18.011" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:48.112" grpc.time_ms=7.446 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.325456] gitaly[897]: time="2026-09-14T08:41:48.136Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:41:58.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:48.125" grpc.time_ms=10.66 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.351532] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbaddb98520d1056c5935ad3","created_at":"2026-09-14T08:41:46.329Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-070c5ac5740b4dba","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:46.332Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3414,"job_status":"rescheduled","message":"PipelineProcessWorker JID-fbaddb98520d1056c5935ad3: rescheduled"} gitlab # [ 424.374951] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.157Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"fbaddb98520d1056c5935ad3","created_at":"2026-09-14T08:41:46.329Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-070c5ac5740b4dba","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:46.332Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-fbaddb98520d1056c5935ad3: done: 0.40814 sec","job_status":"done","queue_duration_s":1.417018,"scheduling_latency_s":1.417042,"gitaly_calls":3,"gitaly_duration_s":0.144499,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.02318,"redis_read_bytes":148,"redis_write_bytes":18014,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.001303,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000991,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.006914,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12313,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.008548,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000361,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001694,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002066,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.066,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.015,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.015,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.034,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.182966,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.35376672,"duration_s":0.40814,"completed_at":"2026-09-14T08:41:48.157Z","load_balancing_strategy":"primary","db_duration_s":0.068145,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.440344] gitaly[897]: time="2026-09-14T08:41:48.138Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8PDJEN7AEH89NQQQ2Q9W grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:18.136" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.137" grpc.time_ms=0.932 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.454256] gitaly[897]: time="2026-09-14T08:41:48.157Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-09-14T08:41:58.036" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:48.156" grpc.time_ms=0.483 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.465912] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.159Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90232a998b69fab0976b6060"],"class":"WebHooks::LogExecutionWorker","jid":"21726dcb508466aa2ca280ab","created_at":"2026-09-14T08:41:47.553Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-b55537a8ae80411b","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:baf560da08beac759f9cd371a87222ff4091cda68194dc7aa7be074a6f399906","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.558Z","job_size_bytes":2591,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-21726dcb508466aa2ca280ab: start","job_status":"start","queue_duration_s":0.601354,"scheduling_latency_s":0.601377} gitlab # [ 424.482765] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.193Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","90232a998b69fab0976b6060"],"class":"WebHooks::LogExecutionWorker","jid":"21726dcb508466aa2ca280ab","created_at":"2026-09-14T08:41:47.553Z","trace_propagation_headers":{"sentry-trace":"b1ddef28b4554026b577c7e9fdc71400-b55537a8ae80411b","baggage":"sentry-trace_id=b1ddef28b4554026b577c7e9fdc71400,sentry-release=Unknown"},"correlation_id":"01M2FH8E0Y8X1DF55H7GBZNFDK","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:baf560da08beac759f9cd371a87222ff4091cda68194dc7aa7be074a6f399906","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.558Z","job_size_bytes":2591,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-21726dcb508466aa2ca280ab: done: 0.034168 sec","job_status":"done","queue_duration_s":0.601354,"scheduling_latency_s":0.601377,"redis_calls":7,"redis_duration_s":0.002071,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001453,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000618,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034168,"completed_at":"2026-09-14T08:41:48.193Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.524945] gitaly[897]: time="2026-09-14T08:41:48.175Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-09-14T08:41:58.041" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:48.161" grpc.time_ms=12.973 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.538067] gitaly[897]: time="2026-09-14T08:41:48.221Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:48.220" grpc.time_ms=0.902 pid=897 span.kind=server system=grpc gitlab # [ 424.547390] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.195Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c4ddd02c048c6da4bb9b6ee6","created_at":"2026-09-14T08:41:47.651Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-6f6f9877f06c4c1e","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.659Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c4ddd02c048c6da4bb9b6ee6: start","job_status":"start","queue_duration_s":0.535782,"scheduling_latency_s":0.535806} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a' gitlab # [ 424.564125] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.290Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"c4ddd02c048c6da4bb9b6ee6","created_at":"2026-09-14T08:41:47.651Z","trace_propagation_headers":{"sentry-trace":"883f3bd1011946efa2b84b6a36b7b543-6f6f9877f06c4c1e","baggage":"sentry-trace_id=883f3bd1011946efa2b84b6a36b7b543,sentry-release=Unknown"},"correlation_id":"01M2FH6021RCZJYXE1RH1W2DG4","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.659Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-c4ddd02c048c6da4bb9b6ee6: done: 0.094982 sec","job_status":"done","queue_duration_s":0.535782,"scheduling_latency_s":0.535806,"redis_calls":11,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000358,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0003,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025342,"duration_s":0.094982,"completed_at":"2026-09-14T08:41:48.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021588,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.605884] gitaly[897]: time="2026-09-14T08:41:48.254Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4156 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:48.253" grpc.time_ms=0.791 pid=897 span.kind=server system=grpc gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a', in 0.06 seconds) (finished: subtest: push delivers the webhook, in 83.53 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 424.615312] gitaly[897]: time="2026-09-14T08:41:48.261Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 diskcache=ba841d48-b5ea-4b6f-a290-4e4b1fc52642 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:41:58.056" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:48.177" pid=897 ref=refs/keep-around/297480606dca023f7b934bf3a4c0ee99c9bd7e3a remote_ip="2001:db8:1::1" revision=297480606dca023f7b934bf3a4c0ee99c9bd7e3a span.kind=server system=grpc username=root gitlab # [ 424.627106] nixbot[3543]: {"ts": "2026-09-14T08:41:48.360Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a \"HTTP/1.1 201 Created\""} gitlab # [ 424.629911] nixbot[3543]: {"ts": "2026-09-14T08:41:48.369Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-297480606dca-5"} gitlab # [ 424.632162] nixbot[3543]: INFO: ::ffff:127.0.0.1:59156 - "GET /api/repos/gitlab/root/test-flake/builds?commit=297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 200 OK gitlab # [ 424.635077] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:48 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 315 gitlab # [ 424.637643] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.291Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8851180a425ad0c8e153ad2f","created_at":"2026-09-14T08:41:47.848Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.851Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8851180a425ad0c8e153ad2f: start","job_status":"start","queue_duration_s":0.440349,"scheduling_latency_s":0.440372} gitlab # [ 424.653608] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.335Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"8851180a425ad0c8e153ad2f","created_at":"2026-09-14T08:41:47.848Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:47.851Z","job_size_bytes":23,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-8851180a425ad0c8e153ad2f: done: 0.044094 sec","job_status":"done","queue_duration_s":0.440349,"scheduling_latency_s":0.440372,"redis_calls":7,"redis_duration_s":0.001092,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000813,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000279,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032575,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.044094,"completed_at":"2026-09-14T08:41:48.335Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008402,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.695422] gitaly[897]: time="2026-09-14T08:41:48.262Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=74 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5469 command.oublock=8 command.real_time_ms=78 command.spawn_token_fork_ms=0 command.system_time_ms=33 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-09-14T08:41:58.056" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-09-14T08:41:48.177" grpc.time_ms=84.736 pid=897 ref=refs/keep-around/297480606dca023f7b934bf3a4c0ee99c9bd7e3a remote_ip="2001:db8:1::1" revision=297480606dca023f7b934bf3a4c0ee99c9bd7e3a span.kind=server system=grpc username=root gitlab # [ 424.709890] nixbot[3543]: {"ts": "2026-09-14T08:41:48.518Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-297480606dca-5", "argv0": "bwrap"} gitlab # [ 424.712264] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.337Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5ddbf1808d7ab17c715547c7","created_at":"2026-09-14T08:41:48.000Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.004Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5ddbf1808d7ab17c715547c7: start","job_status":"start","queue_duration_s":0.333112,"scheduling_latency_s":0.333134} gitlab # [ 424.728438] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.365Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"5ddbf1808d7ab17c715547c7","created_at":"2026-09-14T08:41:48.000Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.004Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-5ddbf1808d7ab17c715547c7: done: 0.027879 sec","job_status":"done","queue_duration_s":0.333112,"scheduling_latency_s":0.333134,"redis_calls":7,"redis_duration_s":0.001142,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000861,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027879,"completed_at":"2026-09-14T08:41:48.364Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.16 seconds) gitlab # [ 424.770863] gitaly[897]: time="2026-09-14T08:41:48.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:18.164" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.264" grpc.time_ms=0.397 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.783060] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:48 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 115 gitlab # [ 424.786093] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.366Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f30b45125d85e761747c1dfb","created_at":"2026-09-14T08:41:48.015Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.018Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f30b45125d85e761747c1dfb: start","job_status":"start","queue_duration_s":0.347512,"scheduling_latency_s":0.347535} gitlab # [ 424.801411] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.397Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"f30b45125d85e761747c1dfb","created_at":"2026-09-14T08:41:48.015Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.018Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-f30b45125d85e761747c1dfb: done: 0.031077 sec","job_status":"done","queue_duration_s":0.347512,"scheduling_latency_s":0.347535,"redis_calls":7,"redis_duration_s":0.0011250000000000001,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000796,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000329,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031077,"completed_at":"2026-09-14T08:41:48.397Z","load_balancing_strategy":"primary","db_duration_s":0.005553,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.843643] gitaly[897]: time="2026-09-14T08:41:48.540Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8RNRGMPD1S94N1HG7BYD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:18.439" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.539" grpc.time_ms=0.534 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.856837] nix-daemon[3720]: accepted connection from pid 4724, user nixbot gitlab # [ 424.857816] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.398Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b5db14d5ac50d9d81fe6f943","created_at":"2026-09-14T08:41:48.031Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.034Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b5db14d5ac50d9d81fe6f943: start","job_status":"start","queue_duration_s":0.364582,"scheduling_latency_s":0.364608} gitlab # [ 424.873020] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.442Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"b5db14d5ac50d9d81fe6f943","created_at":"2026-09-14T08:41:48.031Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.034Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-b5db14d5ac50d9d81fe6f943: done: 0.043882 sec","job_status":"done","queue_duration_s":0.364582,"scheduling_latency_s":0.364608,"redis_calls":10,"redis_duration_s":0.002381,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001498,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000378,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000505,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.043882,"completed_at":"2026-09-14T08:41:48.442Z","load_balancing_strategy":"primary","db_duration_s":0.008315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.915818] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.445Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"591f56bbfff1aa18d750e691","created_at":"2026-09-14T08:41:48.034Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.037Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-591f56bbfff1aa18d750e691: start","job_status":"start","queue_duration_s":0.408334,"scheduling_latency_s":0.408377} gitlab # [ 424.945483] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.491Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"591f56bbfff1aa18d750e691","created_at":"2026-09-14T08:41:48.034Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.037Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-591f56bbfff1aa18d750e691: done: 0.046033 sec","job_status":"done","queue_duration_s":0.408334,"scheduling_latency_s":0.408377,"redis_calls":7,"redis_duration_s":0.002295,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001787,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046033,"completed_at":"2026-09-14T08:41:48.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005753,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.002600] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.493Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"697c90faddc44488d999a1e3","created_at":"2026-09-14T08:41:48.037Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.040Z","job_size_bytes":19,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-697c90faddc44488d999a1e3: start","job_status":"start","queue_duration_s":0.452403,"scheduling_latency_s":0.452428} gitlab # [ 425.021248] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.521Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"697c90faddc44488d999a1e3","created_at":"2026-09-14T08:41:48.037Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.040Z","job_size_bytes":19,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-697c90faddc44488d999a1e3: done: 0.02822 sec","job_status":"done","queue_duration_s":0.452403,"scheduling_latency_s":0.452428,"redis_calls":8,"redis_duration_s":0.001231,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018963,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02822,"completed_at":"2026-09-14T08:41:48.521Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004001,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.064049] gitaly[897]: time="2026-09-14T08:41:48.827Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:18.625" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.826" grpc.time_ms=1.306 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.076961] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.522Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"075021c0a3a73ef431fee7e2","created_at":"2026-09-14T08:41:48.041Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.044Z","job_size_bytes":66,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-075021c0a3a73ef431fee7e2: start","job_status":"start","queue_duration_s":0.478553,"scheduling_latency_s":0.478575} gitlab # [ 425.093846] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.546Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"075021c0a3a73ef431fee7e2","created_at":"2026-09-14T08:41:48.041Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.044Z","job_size_bytes":66,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-075021c0a3a73ef431fee7e2: done: 0.023347 sec","job_status":"done","queue_duration_s":0.478553,"scheduling_latency_s":0.478575,"redis_calls":7,"redis_duration_s":0.001091,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000804,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000287,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023347,"completed_at":"2026-09-14T08:41:48.545Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002449,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.136294] gitaly[897]: time="2026-09-14T08:41:48.875Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8R8BQVMM0J3D3JZFWBA8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:18.673" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:48.874" grpc.time_ms=1.023 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.148262] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.547Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"653ded9afd9a80e05dae2443","created_at":"2026-09-14T08:41:48.044Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.049Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-653ded9afd9a80e05dae2443: start","job_status":"start","queue_duration_s":0.497922,"scheduling_latency_s":0.497941} gitlab # [ 425.164211] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.572Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"653ded9afd9a80e05dae2443","created_at":"2026-09-14T08:41:48.044Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.049Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-653ded9afd9a80e05dae2443: done: 0.025217 sec","job_status":"done","queue_duration_s":0.497922,"scheduling_latency_s":0.497941,"redis_calls":7,"redis_duration_s":0.00101,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00026,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025217,"completed_at":"2026-09-14T08:41:48.572Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004068,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.206094] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.573Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"47ba182a307b210006d80d0f","created_at":"2026-09-14T08:41:48.050Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.053Z","job_size_bytes":6,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-47ba182a307b210006d80d0f: start","job_status":"start","queue_duration_s":0.520282,"scheduling_latency_s":0.520303} gitlab # [ 425.221463] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.587Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"47ba182a307b210006d80d0f","created_at":"2026-09-14T08:41:48.050Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.053Z","job_size_bytes":6,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-47ba182a307b210006d80d0f: done: 0.01367 sec","job_status":"done","queue_duration_s":0.520282,"scheduling_latency_s":0.520303,"redis_calls":7,"redis_duration_s":0.000981,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000715,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000266,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01367,"completed_at":"2026-09-14T08:41:48.587Z","load_balancing_strategy":"primary","db_duration_s":0.000316,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.262197] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.588Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"35b2bf34e9803b9235e27bbb","created_at":"2026-09-14T08:41:48.054Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.057Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-35b2bf34e9803b9235e27bbb: start","job_status":"start","queue_duration_s":0.531329,"scheduling_latency_s":0.531348} gitlab # [ 425.278188] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.612Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"35b2bf34e9803b9235e27bbb","created_at":"2026-09-14T08:41:48.054Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.057Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-35b2bf34e9803b9235e27bbb: done: 0.023418 sec","job_status":"done","queue_duration_s":0.531329,"scheduling_latency_s":0.531348,"redis_calls":7,"redis_duration_s":0.001035,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000319,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023418,"completed_at":"2026-09-14T08:41:48.611Z","load_balancing_strategy":"primary","db_duration_s":0.002907,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.318266] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.613Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9d44e36565a502d0da9ab88e","created_at":"2026-09-14T08:41:48.149Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.151Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9d44e36565a502d0da9ab88e: start","job_status":"start","queue_duration_s":0.461919,"scheduling_latency_s":0.461938} gitlab # [ 425.333632] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.640Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9d44e36565a502d0da9ab88e","created_at":"2026-09-14T08:41:48.149Z","trace_propagation_headers":{"sentry-trace":"bf80c5230bcb451783bc203b74db6efa-e75d4c79a1b542d7","baggage":"sentry-trace_id=bf80c5230bcb451783bc203b74db6efa,sentry-release=Unknown"},"correlation_id":"01M2FH8PDJEN7AEH89NQQQ2Q9W","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.151Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9d44e36565a502d0da9ab88e: done: 0.027348 sec","job_status":"done","queue_duration_s":0.461919,"scheduling_latency_s":0.461938,"redis_calls":8,"redis_duration_s":0.001198,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000897,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000301,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.027348,"completed_at":"2026-09-14T08:41:48.640Z","load_balancing_strategy":"primary","db_duration_s":0.003051,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.374381] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.641Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"49b39863eae852f6fbb5bcde","created_at":"2026-09-14T08:41:48.267Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.270Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-49b39863eae852f6fbb5bcde: start","job_status":"start","queue_duration_s":0.371062,"scheduling_latency_s":0.371078} gitlab # [ 425.390615] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.672Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"49b39863eae852f6fbb5bcde","created_at":"2026-09-14T08:41:48.267Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.270Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-49b39863eae852f6fbb5bcde: done: 0.030562 sec","job_status":"done","queue_duration_s":0.371062,"scheduling_latency_s":0.371078,"redis_calls":7,"redis_duration_s":0.001375,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000445,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02227,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.030562,"completed_at":"2026-09-14T08:41:48.672Z","load_balancing_strategy":"primary","db_duration_s":0.004228,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.432114] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.675Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b3454e11e09be31b56ab4952","created_at":"2026-09-14T08:41:48.271Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.274Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b3454e11e09be31b56ab4952: start","job_status":"start","queue_duration_s":0.401399,"scheduling_latency_s":0.401422} gitlab # [ 425.448943] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.738Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"b3454e11e09be31b56ab4952","created_at":"2026-09-14T08:41:48.271Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.274Z","job_size_bytes":65,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-b3454e11e09be31b56ab4952: done: 0.063111 sec","job_status":"done","queue_duration_s":0.401399,"scheduling_latency_s":0.401422,"redis_calls":10,"redis_duration_s":0.002652,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001675,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042725,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.063111,"completed_at":"2026-09-14T08:41:48.738Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01322,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 425.492163] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.740Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8964ceeefa12714482117c3a","created_at":"2026-09-14T08:41:48.335Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.339Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8964ceeefa12714482117c3a: start","job_status":"start","queue_duration_s":0.401228,"scheduling_latency_s":0.401246} gitlab # [ 425.507590] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.888Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8964ceeefa12714482117c3a","created_at":"2026-09-14T08:41:48.335Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-73dbcc5f87064dea","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.339Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-8964ceeefa12714482117c3a: done: 0.147869 sec","job_status":"done","queue_duration_s":0.401228,"scheduling_latency_s":0.401246,"gitaly_calls":2,"gitaly_duration_s":0.0081,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.007681,"redis_read_bytes":79,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001072,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001293,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000351,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001824,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000358,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000612,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001099,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.038,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.006,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.016,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.099834,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11283122,"duration_s":0.147869,"completed_at":"2026-09-14T08:41:48.888Z","load_balancing_strategy":"primary","db_duration_s":0.040787,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 425.557955] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.889Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a9b6077cb82ef950a9461007","created_at":"2026-09-14T08:41:48.807Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-3668d677dde943fa","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.809Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a9b6077cb82ef950a9461007: start","job_status":"start","queue_duration_s":0.079585,"scheduling_latency_s":0.079604} gitlab # [ 425.573703] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:48.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"a9b6077cb82ef950a9461007","created_at":"2026-09-14T08:41:48.807Z","trace_propagation_headers":{"sentry-trace":"0d8fd940919e4aa2a6c7917e4fffda7f-3668d677dde943fa","baggage":"sentry-trace_id=0d8fd940919e4aa2a6c7917e4fffda7f,sentry-release=Unknown"},"correlation_id":"01M2FH8R8BQVMM0J3D3JZFWBA8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:48.809Z","job_size_bytes":3,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-a9b6077cb82ef950a9461007: done: 0.038621 sec","job_status":"done","queue_duration_s":0.079585,"scheduling_latency_s":0.079604,"redis_calls":7,"redis_duration_s":0.001902,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001365,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000537,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026164,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.038621,"completed_at":"2026-09-14T08:41:48.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008369,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 425.966320] gitaly[897]: time="2026-09-14T08:41:49.794Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8SVWVZ5RG64BZQYF67MM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:19.692" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:49.792" grpc.time_ms=1.155 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.24 seconds) gitlab # [ 426.015684] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:49 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 149 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 427.216144] gitaly[897]: time="2026-09-14T08:41:51.043Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8V1MFBVZMFR7N98H4TZY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:21.040" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:51.041" grpc.time_ms=1.365 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.25 seconds) gitlab # [ 427.269853] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:51 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 196 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 428.457719] gitaly[897]: time="2026-09-14T08:41:52.284Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8W8T3NEY2VPAC1X278CW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:22.182" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:52.283" grpc.time_ms=1.37 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.24 seconds) gitlab # [ 428.514261] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:52 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 186 gitlab # [ 429.483121] nix-daemon[3720]: accepted connection from pid 4755, user nixbot gitlab # [ 429.508709] nix-daemon[3720]: accepted connection from pid 4724, user nixbot gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 429.665878] gitaly[897]: time="2026-09-14T08:41:53.492Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8XFPX46PYQRCDEKS9B4F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:23.390" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:53.491" grpc.time_ms=1.207 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.21 seconds) gitlab # [ 429.723615] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 151 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 430.860726] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:54.687Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"44035a4174a6b5adc1ca565e","created_at":"2026-09-14T08:41:54.612Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.682Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-44035a4174a6b5adc1ca565e: start","job_status":"start","queue_duration_s":0.00439,"scheduling_latency_s":0.004446} gitlab # [ 430.906974] gitaly[897]: time="2026-09-14T08:41:54.733Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8YNH3K6BF8XF8QPNDJ25 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:24.031" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:54.732" grpc.time_ms=1.504 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.28 seconds) gitlab # [ 431.000660] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 217 gitlab # [ 431.722358] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:55.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"44035a4174a6b5adc1ca565e","created_at":"2026-09-14T08:41:54.612Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.682Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-44035a4174a6b5adc1ca565e: done: 0.862531 sec","job_status":"done","queue_duration_s":0.00439,"scheduling_latency_s":0.004446,"redis_calls":70,"redis_duration_s":0.313006,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.03709,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.003928,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.267536,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.004452,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.348928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.862531,"completed_at":"2026-09-14T08:41:55.549Z","load_balancing_strategy":"primary","db_duration_s":0.051778,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.753825] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:55.580Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f876b582f9864235af51fb78","created_at":"2026-09-14T08:41:54.694Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90db566ec7a216f24dcaa70edd0d25c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:41:54.713Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f876b582f9864235af51fb78: start","job_status":"start","queue_duration_s":0.867288,"scheduling_latency_s":0.867344} gitlab # [ 431.820182] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:55.640Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 431.854624] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:55.681Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f876b582f9864235af51fb78","created_at":"2026-09-14T08:41:54.694Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90db566ec7a216f24dcaa70edd0d25c4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-09-14T08:41:54.713Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f876b582f9864235af51fb78: done: 0.101412 sec","job_status":"done","queue_duration_s":0.867288,"scheduling_latency_s":0.867344,"redis_calls":9,"redis_duration_s":0.017945,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002453,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.01086,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004632,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034847,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.101412,"completed_at":"2026-09-14T08:41:55.681Z","load_balancing_strategy":"primary","db_duration_s":0.008935,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.961111] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:55.692Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1ac2befe77c370bc26d7bc91","created_at":"2026-09-14T08:41:54.750Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88bdcab8dc0972c26fa7f55aa4c825ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.774Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1ac2befe77c370bc26d7bc91: start","job_status":"start","queue_duration_s":0.91794,"scheduling_latency_s":0.917982} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 432.159220] gitaly[897]: time="2026-09-14T08:41:55.986Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH8ZXNTBCPXN0Z8PTRJ8DN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:25.284" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:55.985" grpc.time_ms=0.93 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.20 seconds) gitlab # [ 432.200957] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:56 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 133 gitlab # [ 432.382174] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.209Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"1ac2befe77c370bc26d7bc91","created_at":"2026-09-14T08:41:54.750Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88bdcab8dc0972c26fa7f55aa4c825ae","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.774Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-1ac2befe77c370bc26d7bc91: done: 0.517285 sec","job_status":"done","queue_duration_s":0.91794,"scheduling_latency_s":0.917982,"redis_calls":13,"redis_duration_s":0.06486,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0532,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003175,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.114,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.106,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.269809,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00130486,"exclusive_lock_hold_duration_s":0.35180514,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.517285,"completed_at":"2026-09-14T08:41:56.209Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.144574,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.489789] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ff2ad5fb0154ea4facb1c016","created_at":"2026-09-14T08:41:54.793Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9a33eec2ef30f3d7d6d9c5034b89bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.807Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ff2ad5fb0154ea4facb1c016: start","job_status":"start","queue_duration_s":1.403839,"scheduling_latency_s":1.40386} gitlab # [ 432.531392] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"ff2ad5fb0154ea4facb1c016","created_at":"2026-09-14T08:41:54.793Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f9a33eec2ef30f3d7d6d9c5034b89bb3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.807Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-ff2ad5fb0154ea4facb1c016: done: 0.023055 sec","job_status":"done","queue_duration_s":1.403839,"scheduling_latency_s":1.40386,"redis_calls":11,"redis_duration_s":0.002829,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001068,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023055,"completed_at":"2026-09-14T08:41:56.234Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001311,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.600213] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c96ca44a170ca72f8484f212","created_at":"2026-09-14T08:41:54.824Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62a7ae2f9b1ae8ffb2c61e9c50151d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.842Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c96ca44a170ca72f8484f212: start","job_status":"start","queue_duration_s":1.394351,"scheduling_latency_s":1.394367} gitlab # [ 432.615747] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.247Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 432.619475] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.254Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"c96ca44a170ca72f8484f212","created_at":"2026-09-14T08:41:54.824Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62a7ae2f9b1ae8ffb2c61e9c50151d2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.842Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-c96ca44a170ca72f8484f212: done: 0.01787 sec","job_status":"done","queue_duration_s":1.394351,"scheduling_latency_s":1.394367,"redis_calls":9,"redis_duration_s":0.002171,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000481,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001217,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01787,"completed_at":"2026-09-14T08:41:56.254Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.662327] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.255Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"924850ca604ccb5dfbafc269","created_at":"2026-09-14T08:41:54.883Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38714b79ac35d17e228264be172514d6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.915Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-924850ca604ccb5dfbafc269: start","job_status":"start","queue_duration_s":1.339873,"scheduling_latency_s":1.33989} gitlab # [ 432.676418] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.295Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"924850ca604ccb5dfbafc269","created_at":"2026-09-14T08:41:54.883Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"38714b79ac35d17e228264be172514d6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:54.915Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-924850ca604ccb5dfbafc269: done: 0.03959 sec","job_status":"done","queue_duration_s":1.339873,"scheduling_latency_s":1.33989,"redis_calls":9,"redis_duration_s":0.002293,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000463,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.0005,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013319,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.03959,"completed_at":"2026-09-14T08:41:56.295Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019525,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.717435] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c2e72ed70b7df87d6e1cb722","created_at":"2026-09-14T08:41:54.991Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de0924c83ac95cc6377e5328e6a0a159","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:41:55.020Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c2e72ed70b7df87d6e1cb722: start","job_status":"start","queue_duration_s":1.276738,"scheduling_latency_s":1.276754} gitlab # [ 432.730585] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.333Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"c2e72ed70b7df87d6e1cb722","created_at":"2026-09-14T08:41:54.991Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de0924c83ac95cc6377e5328e6a0a159","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-09-14T08:41:55.020Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-c2e72ed70b7df87d6e1cb722: done: 0.0364 sec","job_status":"done","queue_duration_s":1.276738,"scheduling_latency_s":1.276754,"redis_calls":9,"redis_duration_s":0.002409,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000456,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001474,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000479,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019905,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0364,"completed_at":"2026-09-14T08:41:56.333Z","load_balancing_strategy":"primary","db_duration_s":0.009764,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.770870] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bdbdd201fff876e925ef44b3","created_at":"2026-09-14T08:41:55.048Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a232930aefcc8974b2fd8e508bbc7f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.074Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bdbdd201fff876e925ef44b3: start","job_status":"start","queue_duration_s":1.260273,"scheduling_latency_s":1.26029} gitlab # [ 432.784729] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"bdbdd201fff876e925ef44b3","created_at":"2026-09-14T08:41:55.048Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a232930aefcc8974b2fd8e508bbc7f6a","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.074Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-bdbdd201fff876e925ef44b3: done: 0.044722 sec","job_status":"done","queue_duration_s":1.260273,"scheduling_latency_s":1.26029,"redis_calls":9,"redis_duration_s":0.002526,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000468,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001477,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000581,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.018,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.044722,"completed_at":"2026-09-14T08:41:56.379Z","load_balancing_strategy":"primary","db_duration_s":0.02147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.826756] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.381Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8d419670764901081ba23447","created_at":"2026-09-14T08:41:55.110Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a68eaee475a5b3020787f2f80b4c351e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:41:55.137Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8d419670764901081ba23447: start","job_status":"start","queue_duration_s":1.243791,"scheduling_latency_s":1.243812} gitlab # [ 432.840056] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.395Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 432.843536] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"8d419670764901081ba23447","created_at":"2026-09-14T08:41:55.110Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a68eaee475a5b3020787f2f80b4c351e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-09-14T08:41:55.137Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-8d419670764901081ba23447: done: 0.021429 sec","job_status":"done","queue_duration_s":1.243791,"scheduling_latency_s":1.243812,"redis_calls":9,"redis_duration_s":0.002379,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001326,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000516,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021429,"completed_at":"2026-09-14T08:41:56.403Z","load_balancing_strategy":"primary","db_duration_s":0.00067,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.885167] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9066c3a4a8c707fb0fe37117","created_at":"2026-09-14T08:41:55.166Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afd6f8732b699f5d8d17ce987c03652e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.195Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9066c3a4a8c707fb0fe37117: start","job_status":"start","queue_duration_s":1.209692,"scheduling_latency_s":1.209709} gitlab # [ 432.899516] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.416Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 432.902765] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.423Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"9066c3a4a8c707fb0fe37117","created_at":"2026-09-14T08:41:55.166Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"afd6f8732b699f5d8d17ce987c03652e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.195Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-9066c3a4a8c707fb0fe37117: done: 0.01857 sec","job_status":"done","queue_duration_s":1.209692,"scheduling_latency_s":1.209709,"redis_calls":9,"redis_duration_s":0.002137,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000462,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001211,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010788,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01857,"completed_at":"2026-09-14T08:41:56.423Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.944415] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.424Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"74a0bece09df896662ce5c96","created_at":"2026-09-14T08:41:55.225Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65f0086e18e05322f2a7997e8cdce854","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.251Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-74a0bece09df896662ce5c96: start","job_status":"start","queue_duration_s":1.173756,"scheduling_latency_s":1.173773} gitlab # [ 432.957823] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"74a0bece09df896662ce5c96","created_at":"2026-09-14T08:41:55.225Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65f0086e18e05322f2a7997e8cdce854","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.251Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-74a0bece09df896662ce5c96: done: 0.034034 sec","job_status":"done","queue_duration_s":1.173756,"scheduling_latency_s":1.173773,"redis_calls":18,"redis_duration_s":0.004187,"redis_read_bytes":20,"redis_write_bytes":4702,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000921,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000331,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.002206,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000729,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022867,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034034,"completed_at":"2026-09-14T08:41:56.458Z","load_balancing_strategy":"primary","db_duration_s":0.003147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.000127] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f428d6b26a0736bf36de6238","created_at":"2026-09-14T08:41:55.274Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33260820b405a4756de3cc7ee7f7e58b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.301Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f428d6b26a0736bf36de6238: start","job_status":"start","queue_duration_s":1.159712,"scheduling_latency_s":1.159755} gitlab # [ 433.013817] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.515Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"f428d6b26a0736bf36de6238","created_at":"2026-09-14T08:41:55.274Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"33260820b405a4756de3cc7ee7f7e58b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.301Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-f428d6b26a0736bf36de6238: done: 0.054085 sec","job_status":"done","queue_duration_s":1.159712,"scheduling_latency_s":1.159755,"redis_calls":9,"redis_duration_s":0.002999,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001707,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000721,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034329,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.054085,"completed_at":"2026-09-14T08:41:56.515Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004915,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.055756] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.518Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0a20318cd243bd904e5d867d","created_at":"2026-09-14T08:41:55.324Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6cc6fbee7b11fa08b46134043d1c38","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.365Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0a20318cd243bd904e5d867d: start","job_status":"start","queue_duration_s":1.152702,"scheduling_latency_s":1.152744} gitlab # [ 433.070917] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"0a20318cd243bd904e5d867d","created_at":"2026-09-14T08:41:55.324Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6cc6fbee7b11fa08b46134043d1c38","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.365Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-0a20318cd243bd904e5d867d: done: 0.043447 sec","job_status":"done","queue_duration_s":1.152702,"scheduling_latency_s":1.152744,"redis_calls":11,"redis_duration_s":0.004243,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000629,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001719,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001895,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00094978,"exclusive_lock_hold_duration_s":0.00540438,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.043447,"completed_at":"2026-09-14T08:41:56.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002887,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.114309] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d67ed168d04c92bad4fb8945","created_at":"2026-09-14T08:41:55.413Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f31a15287e10c87769a12271fc1624dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.443Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d67ed168d04c92bad4fb8945: start","job_status":"start","queue_duration_s":1.120985,"scheduling_latency_s":1.121013} gitlab # [ 433.129010] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d67ed168d04c92bad4fb8945","created_at":"2026-09-14T08:41:55.413Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-cb76a065e1044c3a","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f31a15287e10c87769a12271fc1624dc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.443Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d67ed168d04c92bad4fb8945: done: 0.028621 sec","job_status":"done","queue_duration_s":1.120985,"scheduling_latency_s":1.121013,"redis_calls":9,"redis_duration_s":0.002782,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000581,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00161,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017222,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028621,"completed_at":"2026-09-14T08:41:56.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000767,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.170869] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.594Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7553b5c9821f7a3850886e21","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7553b5c9821f7a3850886e21: start","job_status":"start","queue_duration_s":1.087574,"scheduling_latency_s":1.087596} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 433.183311] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7553b5c9821f7a3850886e21","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7553b5c9821f7a3850886e21: done: 0.043656 sec","job_status":"done","queue_duration_s":1.087574,"scheduling_latency_s":1.087596,"redis_calls":15,"redis_duration_s":0.007427,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000531,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.005336,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026271,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.043656,"completed_at":"2026-09-14T08:41:56.638Z","load_balancing_strategy":"primary","db_duration_s":0.000698,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.222167] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.641Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"777b4ac1c68e4de52ab8862b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-777b4ac1c68e4de52ab8862b: start","job_status":"start","queue_duration_s":1.134185,"scheduling_latency_s":1.134224} gitlab # [ 433.233713] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.680Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"777b4ac1c68e4de52ab8862b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-777b4ac1c68e4de52ab8862b: done: 0.039585 sec","job_status":"done","queue_duration_s":1.134185,"scheduling_latency_s":1.134224,"redis_calls":13,"redis_duration_s":0.00543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003884,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027186,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.039585,"completed_at":"2026-09-14T08:41:56.680Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.273084] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.683Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"919b33f28693ef8ed50a5be6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-919b33f28693ef8ed50a5be6: start","job_status":"start","queue_duration_s":1.176759,"scheduling_latency_s":1.176794} gitlab # [ 433.286057] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.711Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"919b33f28693ef8ed50a5be6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-919b33f28693ef8ed50a5be6: done: 0.027809 sec","job_status":"done","queue_duration_s":1.176759,"scheduling_latency_s":1.176794,"redis_calls":13,"redis_duration_s":0.003769,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001265,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002504,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.027809,"completed_at":"2026-09-14T08:41:56.711Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.324689] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daf7e933e5cfc60abc76409c","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daf7e933e5cfc60abc76409c: start","job_status":"start","queue_duration_s":1.206671,"scheduling_latency_s":1.206698} gitlab # [ 433.336851] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"daf7e933e5cfc60abc76409c","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-daf7e933e5cfc60abc76409c: done: 0.023258 sec","job_status":"done","queue_duration_s":1.206671,"scheduling_latency_s":1.206698,"redis_calls":13,"redis_duration_s":0.003352,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001062,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01516,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023258,"completed_at":"2026-09-14T08:41:56.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.375784] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a1f5af8b86a24ba622b7c74","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a1f5af8b86a24ba622b7c74: start","job_status":"start","queue_duration_s":1.231762,"scheduling_latency_s":1.231782} gitlab # [ 433.388295] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.757Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7a1f5af8b86a24ba622b7c74","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7a1f5af8b86a24ba622b7c74: done: 0.018914 sec","job_status":"done","queue_duration_s":1.231762,"scheduling_latency_s":1.231782,"redis_calls":13,"redis_duration_s":0.003097,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001026,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002071,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011833,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018914,"completed_at":"2026-09-14T08:41:56.757Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.426541] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e37839b9b5cc137b27695a6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e37839b9b5cc137b27695a6: start","job_status":"start","queue_duration_s":1.252239,"scheduling_latency_s":1.252256} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.26 seconds) gitlab # [ 433.438916] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.775Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9e37839b9b5cc137b27695a6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9e37839b9b5cc137b27695a6: done: 0.016481 sec","job_status":"done","queue_duration_s":1.252239,"scheduling_latency_s":1.252256,"redis_calls":13,"redis_duration_s":0.0030139999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000993,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010248,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016481,"completed_at":"2026-09-14T08:41:56.775Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.477052] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"947a99e1ee8ec66e81b9ff86","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-947a99e1ee8ec66e81b9ff86: start","job_status":"start","queue_duration_s":1.270125,"scheduling_latency_s":1.27014} gitlab # [ 433.490563] gitaly[897]: time="2026-09-14T08:41:57.223Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH913STZ86QZ27BVK36KT5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:27.020" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:57.221" grpc.time_ms=1.323 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 433.503292] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:57 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 163 gitlab # [ 433.506082] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.792Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"947a99e1ee8ec66e81b9ff86","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-947a99e1ee8ec66e81b9ff86: done: 0.014912 sec","job_status":"done","queue_duration_s":1.270125,"scheduling_latency_s":1.27014,"redis_calls":13,"redis_duration_s":0.002853,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000973,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009147,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014912,"completed_at":"2026-09-14T08:41:56.792Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.544799] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.793Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23eb00021a3e732672f07ae7","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23eb00021a3e732672f07ae7: start","job_status":"start","queue_duration_s":1.286319,"scheduling_latency_s":1.286334} gitlab # [ 433.556809] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.812Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"23eb00021a3e732672f07ae7","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-23eb00021a3e732672f07ae7: done: 0.019305 sec","job_status":"done","queue_duration_s":1.286319,"scheduling_latency_s":1.286334,"redis_calls":13,"redis_duration_s":0.0031609999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001234,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011834,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.019305,"completed_at":"2026-09-14T08:41:56.812Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.595281] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0173dd894bddd23f1e830cd6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0173dd894bddd23f1e830cd6: start","job_status":"start","queue_duration_s":1.308501,"scheduling_latency_s":1.30854} gitlab # [ 433.607706] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0173dd894bddd23f1e830cd6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0173dd894bddd23f1e830cd6: done: 0.035409 sec","job_status":"done","queue_duration_s":1.308501,"scheduling_latency_s":1.30854,"redis_calls":13,"redis_duration_s":0.004611,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001714,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.035409,"completed_at":"2026-09-14T08:41:56.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.646433] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c4f2648747488729b6d4a2b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c4f2648747488729b6d4a2b: start","job_status":"start","queue_duration_s":1.347127,"scheduling_latency_s":1.347162} gitlab # [ 433.658797] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.882Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7c4f2648747488729b6d4a2b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7c4f2648747488729b6d4a2b: done: 0.028462 sec","job_status":"done","queue_duration_s":1.347127,"scheduling_latency_s":1.347162,"redis_calls":13,"redis_duration_s":0.004021,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001336,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002685,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018672,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028462,"completed_at":"2026-09-14T08:41:56.882Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.698313] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.884Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ca232604326ad51e65a3558","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ca232604326ad51e65a3558: start","job_status":"start","queue_duration_s":1.377699,"scheduling_latency_s":1.377725} gitlab # [ 433.710579] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ca232604326ad51e65a3558","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ca232604326ad51e65a3558: done: 0.022051 sec","job_status":"done","queue_duration_s":1.377699,"scheduling_latency_s":1.377725,"redis_calls":13,"redis_duration_s":0.0033309999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001132,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002199,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022051,"completed_at":"2026-09-14T08:41:56.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.749212] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ccb20206bcdb43bdef7e2b9","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ccb20206bcdb43bdef7e2b9: start","job_status":"start","queue_duration_s":1.401436,"scheduling_latency_s":1.401457} gitlab # [ 433.761382] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.926Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4ccb20206bcdb43bdef7e2b9","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4ccb20206bcdb43bdef7e2b9: done: 0.018266 sec","job_status":"done","queue_duration_s":1.401436,"scheduling_latency_s":1.401457,"redis_calls":13,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001028,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002019,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011628,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018266,"completed_at":"2026-09-14T08:41:56.926Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.800262] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.928Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2a5c9de3e0afa56df6eeef8","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2a5c9de3e0afa56df6eeef8: start","job_status":"start","queue_duration_s":1.421261,"scheduling_latency_s":1.421277} gitlab # [ 433.812535] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.943Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2a5c9de3e0afa56df6eeef8","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2a5c9de3e0afa56df6eeef8: done: 0.015402 sec","job_status":"done","queue_duration_s":1.421261,"scheduling_latency_s":1.421277,"redis_calls":13,"redis_duration_s":0.002503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000941,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001562,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015402,"completed_at":"2026-09-14T08:41:56.943Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.851051] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69818c78cdf99ef12f1a4039","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69818c78cdf99ef12f1a4039: start","job_status":"start","queue_duration_s":1.437968,"scheduling_latency_s":1.437983} gitlab # [ 433.863315] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.960Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69818c78cdf99ef12f1a4039","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69818c78cdf99ef12f1a4039: done: 0.01489 sec","job_status":"done","queue_duration_s":1.437968,"scheduling_latency_s":1.437983,"redis_calls":13,"redis_duration_s":0.002816,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00096,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001856,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009249,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01489,"completed_at":"2026-09-14T08:41:56.959Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.903217] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35306080b649338f5a49da48","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35306080b649338f5a49da48: start","job_status":"start","queue_duration_s":1.454167,"scheduling_latency_s":1.454181} gitlab # [ 433.915609] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"35306080b649338f5a49da48","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-35306080b649338f5a49da48: done: 0.014503 sec","job_status":"done","queue_duration_s":1.454167,"scheduling_latency_s":1.454181,"redis_calls":13,"redis_duration_s":0.002745,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000911,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001834,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014503,"completed_at":"2026-09-14T08:41:56.975Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.954528] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4ed6475b001fee0f7a5add6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4ed6475b001fee0f7a5add6: start","job_status":"start","queue_duration_s":1.470103,"scheduling_latency_s":1.470118} gitlab # [ 433.966843] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e4ed6475b001fee0f7a5add6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e4ed6475b001fee0f7a5add6: done: 0.014179 sec","job_status":"done","queue_duration_s":1.470103,"scheduling_latency_s":1.470118,"redis_calls":13,"redis_duration_s":0.002645,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000903,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001742,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014179,"completed_at":"2026-09-14T08:41:56.991Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.005771] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:56.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6bcc561c664062cb88a9f00","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6bcc561c664062cb88a9f00: start","job_status":"start","queue_duration_s":1.485715,"scheduling_latency_s":1.485729} gitlab # [ 434.017778] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e6bcc561c664062cb88a9f00","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e6bcc561c664062cb88a9f00: done: 0.099914 sec","job_status":"done","queue_duration_s":1.485715,"scheduling_latency_s":1.485729,"redis_calls":13,"redis_duration_s":0.0029059999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000943,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001963,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.093641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.099914,"completed_at":"2026-09-14T08:41:57.092Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.056092] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.093Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185b1e510a335d0c3597bb5e","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185b1e510a335d0c3597bb5e: start","job_status":"start","queue_duration_s":1.586788,"scheduling_latency_s":1.586804} gitlab # [ 434.068277] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185b1e510a335d0c3597bb5e","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185b1e510a335d0c3597bb5e: done: 0.01635 sec","job_status":"done","queue_duration_s":1.586788,"scheduling_latency_s":1.586804,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000996,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011922,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01635,"completed_at":"2026-09-14T08:41:57.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.107588] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fc9a920108074ca726b4753","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fc9a920108074ca726b4753: start","job_status":"start","queue_duration_s":1.60432,"scheduling_latency_s":1.604337} gitlab # [ 434.119618] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.125Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2fc9a920108074ca726b4753","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2fc9a920108074ca726b4753: done: 0.013597 sec","job_status":"done","queue_duration_s":1.60432,"scheduling_latency_s":1.604337,"redis_calls":13,"redis_duration_s":0.0015,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009263,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013597,"completed_at":"2026-09-14T08:41:57.125Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.158304] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef70b79b95d177f250301f9e","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef70b79b95d177f250301f9e: start","job_status":"start","queue_duration_s":1.619685,"scheduling_latency_s":1.619699} gitlab # [ 434.170377] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ef70b79b95d177f250301f9e","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ef70b79b95d177f250301f9e: done: 0.014295 sec","job_status":"done","queue_duration_s":1.619685,"scheduling_latency_s":1.619699,"redis_calls":13,"redis_duration_s":0.001663,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00113,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009921,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014295,"completed_at":"2026-09-14T08:41:57.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.208910] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5254df628ae741d234e66f2","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5254df628ae741d234e66f2: start","job_status":"start","queue_duration_s":1.634992,"scheduling_latency_s":1.635008} gitlab # [ 434.220851] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5254df628ae741d234e66f2","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5254df628ae741d234e66f2: done: 0.014042 sec","job_status":"done","queue_duration_s":1.634992,"scheduling_latency_s":1.635008,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000909,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009915,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014042,"completed_at":"2026-09-14T08:41:57.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.259606] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c216989483092f0e269d66a","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c216989483092f0e269d66a: start","job_status":"start","queue_duration_s":1.65007,"scheduling_latency_s":1.650087} gitlab # [ 434.276591] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0c216989483092f0e269d66a","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0c216989483092f0e269d66a: done: 0.01481 sec","job_status":"done","queue_duration_s":1.65007,"scheduling_latency_s":1.650087,"redis_calls":13,"redis_duration_s":0.0014060000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01481,"completed_at":"2026-09-14T08:41:57.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.347830] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.173Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb9ada50dc6e178a7859dc59","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb9ada50dc6e178a7859dc59: start","job_status":"start","queue_duration_s":1.666002,"scheduling_latency_s":1.666017} gitlab # [ 434.363217] nix-daemon[3720]: accepted connection from pid 4724, user nixbot gitlab # [ 434.364305] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cb9ada50dc6e178a7859dc59","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cb9ada50dc6e178a7859dc59: done: 0.014223 sec","job_status":"done","queue_duration_s":1.666002,"scheduling_latency_s":1.666017,"redis_calls":13,"redis_duration_s":0.001465,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000926,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01003,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014223,"completed_at":"2026-09-14T08:41:57.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.406816] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.188Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0537306ae9ff78f823e0d59a","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0537306ae9ff78f823e0d59a: start","job_status":"start","queue_duration_s":1.681212,"scheduling_latency_s":1.681226} gitlab # [ 434.420533] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.202Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0537306ae9ff78f823e0d59a","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0537306ae9ff78f823e0d59a: done: 0.014566 sec","job_status":"done","queue_duration_s":1.681212,"scheduling_latency_s":1.681226,"redis_calls":13,"redis_duration_s":0.001373,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0005,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000873,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009735,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014566,"completed_at":"2026-09-14T08:41:57.202Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 434.459623] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.203Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87a1fdc37c6cf679544ebe35","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87a1fdc37c6cf679544ebe35: start","job_status":"start","queue_duration_s":1.696808,"scheduling_latency_s":1.696824} gitlab # [ 434.471526] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"87a1fdc37c6cf679544ebe35","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-87a1fdc37c6cf679544ebe35: done: 0.016413 sec","job_status":"done","queue_duration_s":1.696808,"scheduling_latency_s":1.696824,"redis_calls":13,"redis_duration_s":0.003256,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.002265,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010486,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016413,"completed_at":"2026-09-14T08:41:57.220Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.509244] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.221Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac4fcbcdf6af4cd9ecb2b291","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac4fcbcdf6af4cd9ecb2b291: start","job_status":"start","queue_duration_s":1.714302,"scheduling_latency_s":1.714317} gitlab # [ 434.521303] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.235Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ac4fcbcdf6af4cd9ecb2b291","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ac4fcbcdf6af4cd9ecb2b291: done: 0.014203 sec","job_status":"done","queue_duration_s":1.714302,"scheduling_latency_s":1.714317,"redis_calls":13,"redis_duration_s":0.001392,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000886,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014203,"completed_at":"2026-09-14T08:41:57.235Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.560472] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.237Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"294b152a62ecb1eb5d06fac3","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-294b152a62ecb1eb5d06fac3: start","job_status":"start","queue_duration_s":1.730221,"scheduling_latency_s":1.730236} gitlab # [ 434.572990] gitaly[897]: time="2026-09-14T08:41:58.324Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:28.122" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.323" grpc.time_ms=1.055 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.585831] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.251Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"294b152a62ecb1eb5d06fac3","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-294b152a62ecb1eb5d06fac3: done: 0.013863 sec","job_status":"done","queue_duration_s":1.730221,"scheduling_latency_s":1.730236,"redis_calls":13,"redis_duration_s":0.001961,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001053,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000908,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009692,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013863,"completed_at":"2026-09-14T08:41:57.251Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.624657] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.252Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b298614f0bd491e6b80a901d","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b298614f0bd491e6b80a901d: start","job_status":"start","queue_duration_s":1.745175,"scheduling_latency_s":1.745191} gitlab # [ 434.637334] gitaly[897]: time="2026-09-14T08:41:58.412Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:42:08.003" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.403" grpc.time_ms=7.147 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.651397] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b298614f0bd491e6b80a901d","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b298614f0bd491e6b80a901d: done: 0.014128 sec","job_status":"done","queue_duration_s":1.745175,"scheduling_latency_s":1.745191,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014128,"completed_at":"2026-09-14T08:41:57.266Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.690772] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.267Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dbf9013d7a24b0601e5c33c","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dbf9013d7a24b0601e5c33c: start","job_status":"start","queue_duration_s":1.760406,"scheduling_latency_s":1.760421} gitlab # [ 434.703593] gitaly[897]: time="2026-09-14T08:41:58.469Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92BGAYAXGSFRQT7HZHH5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:28.368" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.468" grpc.time_ms=0.822 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.717889] nixbot[3543]: {"ts": "2026-09-14T08:41:58.479Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a \"HTTP/1.1 201 Created\""} gitlab # [ 434.720943] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 254 gitlab # [ 434.723402] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0dbf9013d7a24b0601e5c33c","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0dbf9013d7a24b0601e5c33c: done: 0.013825 sec","job_status":"done","queue_duration_s":1.760406,"scheduling_latency_s":1.760421,"redis_calls":13,"redis_duration_s":0.00138,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000499,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000881,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013825,"completed_at":"2026-09-14T08:41:57.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.761803] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.283Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f4d77c24beec06cab972079","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f4d77c24beec06cab972079: start","job_status":"start","queue_duration_s":1.776449,"scheduling_latency_s":1.776465} gitlab # [ 434.774089] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4f4d77c24beec06cab972079","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4f4d77c24beec06cab972079: done: 0.015368 sec","job_status":"done","queue_duration_s":1.776449,"scheduling_latency_s":1.776465,"redis_calls":13,"redis_duration_s":0.002819,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000983,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009554,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015368,"completed_at":"2026-09-14T08:41:57.298Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.812645] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.300Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"982e6ccc0890bd1a45c943f6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-982e6ccc0890bd1a45c943f6: start","job_status":"start","queue_duration_s":1.793145,"scheduling_latency_s":1.79316} gitlab # [ 434.824792] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.316Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"982e6ccc0890bd1a45c943f6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-982e6ccc0890bd1a45c943f6: done: 0.015886 sec","job_status":"done","queue_duration_s":1.793145,"scheduling_latency_s":1.79316,"redis_calls":13,"redis_duration_s":0.0029790000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000965,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015886,"completed_at":"2026-09-14T08:41:57.316Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.863855] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.317Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30eb613896883912716fe03d","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30eb613896883912716fe03d: start","job_status":"start","queue_duration_s":1.810272,"scheduling_latency_s":1.810287} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.42 seconds) gitlab # [ 434.875721] gitaly[897]: time="2026-09-14T08:41:58.603Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:58.603" grpc.time_ms=0.492 pid=897 span.kind=server system=grpc gitlab # [ 434.885262] gitaly[897]: time="2026-09-14T08:41:58.636Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:58.635" grpc.time_ms=0.458 pid=897 span.kind=server system=grpc gitlab # [ 434.894752] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:58 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses HTTP/1.1" 200 677 "" "curl/8.21.0" 306 gitlab # [ 434.897441] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.332Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"30eb613896883912716fe03d","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-30eb613896883912716fe03d: done: 0.014801 sec","job_status":"done","queue_duration_s":1.810272,"scheduling_latency_s":1.810287,"redis_calls":13,"redis_duration_s":0.002754,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000905,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001849,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008671,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014801,"completed_at":"2026-09-14T08:41:57.332Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 434.936909] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.334Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac3575b4c8cd9b89e278b17","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac3575b4c8cd9b89e278b17: start","job_status":"start","queue_duration_s":1.827334,"scheduling_latency_s":1.82735} gitlab # [ 434.948778] gitaly[897]: time="2026-09-14T08:41:58.673Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:58.672" grpc.time_ms=0.338 pid=897 span.kind=server system=grpc gitlab # [ 434.958204] gitaly[897]: time="2026-09-14T08:41:58.677Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" operations="[repack_refs]" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 434.968000] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.350Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ac3575b4c8cd9b89e278b17","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ac3575b4c8cd9b89e278b17: done: 0.015542 sec","job_status":"done","queue_duration_s":1.827334,"scheduling_latency_s":1.82735,"redis_calls":13,"redis_duration_s":0.002635,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001681,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010102,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015542,"completed_at":"2026-09-14T08:41:57.349Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.006647] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.351Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"156a443862e0950f5ca2c479","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-156a443862e0950f5ca2c479: start","job_status":"start","queue_duration_s":1.844204,"scheduling_latency_s":1.844218} gitlab # [ 435.018973] gitaly[897]: time="2026-09-14T08:41:58.679Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" pid=897 remote_ip="2001:db8:1::1" repository_info="{false {6 704 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-09-14 08:40:09.41146152 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 435.031007] gitaly[897]: time="2026-09-14T08:41:58.684Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM diskcache=20093560-525e-4b39-a6c3-9678bfddff61 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.040794] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.367Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"156a443862e0950f5ca2c479","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-156a443862e0950f5ca2c479: done: 0.016075 sec","job_status":"done","queue_duration_s":1.844204,"scheduling_latency_s":1.844218,"redis_calls":13,"redis_duration_s":0.0028840000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000907,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016075,"completed_at":"2026-09-14T08:41:57.367Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.079555] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.368Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"377e26d9cd92c19cb2766df7","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-377e26d9cd92c19cb2766df7: start","job_status":"start","queue_duration_s":1.861725,"scheduling_latency_s":1.861739} gitlab # [ 435.092109] gitaly[897]: time="2026-09-14T08:41:58.684Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7159 command.oublock=0 command.real_time_ms=101 command.spawn_token_fork_ms=2 command.system_time_ms=53 command.user_time_ms=24 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" grpc.time_ms=110.054 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.104724] nixbot[3543]: {"ts": "2026-09-14T08:41:58.906Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a \"HTTP/1.1 201 Created\""} gitlab # [ 435.107341] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.384Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"377e26d9cd92c19cb2766df7","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-377e26d9cd92c19cb2766df7: done: 0.015974 sec","job_status":"done","queue_duration_s":1.861725,"scheduling_latency_s":1.861739,"redis_calls":13,"redis_duration_s":0.002801,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000912,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001889,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009726,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015974,"completed_at":"2026-09-14T08:41:57.384Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.147084] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.386Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b20a1d5a4534ca4e4b8fcc98","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b20a1d5a4534ca4e4b8fcc98: start","job_status":"start","queue_duration_s":1.879099,"scheduling_latency_s":1.879113} gitlab # [ 435.159407] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:58 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 419 gitlab # [ 435.162000] gitaly[897]: time="2026-09-14T08:41:58.703Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" pid=897 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root gitlab # [ 435.175064] gitaly[897]: time="2026-09-14T08:41:58.704Z" level=info msg="optimized repository" clean_stale_data_time=24.01374ms clean_work_trees_time="93.34µs" commit_graph_time=720ns correlation_id=01M2FH926EZA92VQS46NWM06QM gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="1.38µs" pid=897 prune_time=260ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.36µs" storage_name=default system=housekeeping total_time=24.15136ms gitlab # [ 435.180931] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b20a1d5a4534ca4e4b8fcc98","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b20a1d5a4534ca4e4b8fcc98: done: 0.015718 sec","job_status":"done","queue_duration_s":1.879099,"scheduling_latency_s":1.879113,"redis_calls":13,"redis_duration_s":0.003763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00092,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002843,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015718,"completed_at":"2026-09-14T08:41:57.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.219226] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d7c0261015287dff7c24030","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d7c0261015287dff7c24030: start","job_status":"start","queue_duration_s":1.896303,"scheduling_latency_s":1.896318} gitlab # [ 435.231534] gitaly[897]: time="2026-09-14T08:41:58.704Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:28.374" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.574" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.240589] gitaly[897]: time="2026-09-14T08:41:58.753Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:28.552" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.752" grpc.time_ms=0.676 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.253228] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.419Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6d7c0261015287dff7c24030","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6d7c0261015287dff7c24030: done: 0.015589 sec","job_status":"done","queue_duration_s":1.896303,"scheduling_latency_s":1.896318,"redis_calls":13,"redis_duration_s":0.002802,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00188,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015589,"completed_at":"2026-09-14T08:41:57.419Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.292133] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.421Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2507ba63e9e0ce2944bb8e18","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2507ba63e9e0ce2944bb8e18: start","job_status":"start","queue_duration_s":1.914438,"scheduling_latency_s":1.914453} gitlab # [ 435.304207] gitaly[897]: time="2026-09-14T08:41:58.793Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH926EZA92VQS46NWM06QM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:28.590" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.792" grpc.time_ms=1.033 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.316548] nixbot[3543]: {"ts": "2026-09-14T08:41:59.115Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a \"HTTP/1.1 201 Created\""} gitlab # [ 435.319247] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.437Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2507ba63e9e0ce2944bb8e18","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2507ba63e9e0ce2944bb8e18: done: 0.015693 sec","job_status":"done","queue_duration_s":1.914438,"scheduling_latency_s":1.914453,"redis_calls":13,"redis_duration_s":0.002778,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000937,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001841,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015693,"completed_at":"2026-09-14T08:41:57.437Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.359577] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.438Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fefc0515b6b44a64675e1ce4","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fefc0515b6b44a64675e1ce4: start","job_status":"start","queue_duration_s":1.931484,"scheduling_latency_s":1.931499} gitlab # [ 435.372023] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:59 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/297480606dca023f7b934bf3a4c0ee99c9bd7e3a HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 194 gitlab # [ 435.374419] gitaly[897]: time="2026-09-14T08:41:58.799Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:28.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.798" grpc.time_ms=0.744 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.387060] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.454Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fefc0515b6b44a64675e1ce4","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fefc0515b6b44a64675e1ce4: done: 0.015956 sec","job_status":"done","queue_duration_s":1.931484,"scheduling_latency_s":1.931499,"redis_calls":13,"redis_duration_s":0.00265,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000901,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001749,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015956,"completed_at":"2026-09-14T08:41:57.454Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.425739] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"368d2c4a58dbdd352b17baef","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-368d2c4a58dbdd352b17baef: start","job_status":"start","queue_duration_s":1.948774,"scheduling_latency_s":1.948788} gitlab # [ 435.438018] gitaly[897]: time="2026-09-14T08:41:58.846Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:42:08.198" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:58.838" grpc.time_ms=7.991 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.451165] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"368d2c4a58dbdd352b17baef","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-368d2c4a58dbdd352b17baef: done: 0.015261 sec","job_status":"done","queue_duration_s":1.948774,"scheduling_latency_s":1.948788,"redis_calls":13,"redis_duration_s":0.003715,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000924,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002791,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009677,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015261,"completed_at":"2026-09-14T08:41:57.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.490623] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b7cd02da1df08c3517324b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b7cd02da1df08c3517324b: start","job_status":"start","queue_duration_s":1.96538,"scheduling_latency_s":1.965394} gitlab # [ 435.503179] gitaly[897]: time="2026-09-14T08:41:58.980Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92W8XND216Y72QFNVXKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:28.879" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:58.979" grpc.time_ms=0.674 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.515944] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"16b7cd02da1df08c3517324b","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-16b7cd02da1df08c3517324b: done: 0.016391 sec","job_status":"done","queue_duration_s":1.96538,"scheduling_latency_s":1.965394,"redis_calls":13,"redis_duration_s":0.002763,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001812,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009975,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016391,"completed_at":"2026-09-14T08:41:57.488Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.556230] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.490Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d53ab5ad0e9453ce6fe551fb","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d53ab5ad0e9453ce6fe551fb: start","job_status":"start","queue_duration_s":1.983086,"scheduling_latency_s":1.9831} gitlab # [ 435.568493] gitaly[897]: time="2026-09-14T08:41:59.033Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=9 component=gitaly.StreamServerInterceptor correlation_id=01M2FH92W8XND216Y72QFNVXKC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-09-14T08:42:09.006" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:59.016" grpc.time_ms=17.204 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.581586] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.505Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d53ab5ad0e9453ce6fe551fb","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d53ab5ad0e9453ce6fe551fb: done: 0.015638 sec","job_status":"done","queue_duration_s":1.983086,"scheduling_latency_s":1.9831,"redis_calls":13,"redis_duration_s":0.002785,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001868,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008693,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015638,"completed_at":"2026-09-14T08:41:57.505Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.620348] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.507Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfd0b3f9ac3ee1a6aa750ccb","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfd0b3f9ac3ee1a6aa750ccb: start","job_status":"start","queue_duration_s":2.000175,"scheduling_latency_s":2.000189} gitlab # [ 435.633024] gitaly[897]: time="2026-09-14T08:41:59.398Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:59.397" grpc.time_ms=0.861 pid=897 span.kind=server system=grpc gitlab # [ 435.642302] gitaly[897]: time="2026-09-14T08:41:59.432Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:59.431" grpc.time_ms=0.522 pid=897 span.kind=server system=grpc gitlab # [ 435.652703] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.522Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"dfd0b3f9ac3ee1a6aa750ccb","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-dfd0b3f9ac3ee1a6aa750ccb: done: 0.014936 sec","job_status":"done","queue_duration_s":2.000175,"scheduling_latency_s":2.000189,"redis_calls":13,"redis_duration_s":0.00281,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00098,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00183,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009379,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014936,"completed_at":"2026-09-14T08:41:57.522Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.692643] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.523Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e03e6cfeec19db5fac9351","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e03e6cfeec19db5fac9351: start","job_status":"start","queue_duration_s":2.016487,"scheduling_latency_s":2.016501} gitlab # [ 435.704698] gitaly[897]: time="2026-09-14T08:41:59.472Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3898 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-09-14T08:41:59.470" grpc.time_ms=1.416 pid=897 span.kind=server system=grpc gitlab # [ 435.714298] gitaly[897]: time="2026-09-14T08:41:59.480Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC diskcache=c12bb89d-22d8-4bc4-a393-ebf5bd3b9b6e grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:29.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:59.357" pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.724257] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f1e03e6cfeec19db5fac9351","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f1e03e6cfeec19db5fac9351: done: 0.014993 sec","job_status":"done","queue_duration_s":2.016487,"scheduling_latency_s":2.016501,"redis_calls":13,"redis_duration_s":0.002725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000951,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001774,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008971,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014993,"completed_at":"2026-09-14T08:41:57.538Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.764400] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.540Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"031b241829bfecad51883231","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-031b241829bfecad51883231: start","job_status":"start","queue_duration_s":2.03287,"scheduling_latency_s":2.032884} gitlab # [ 435.776432] gitaly[897]: time="2026-09-14T08:41:59.480Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=120 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7213 command.oublock=0 command.real_time_ms=117 command.spawn_token_fork_ms=1 command.system_time_ms=65 command.user_time_ms=55 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-09-14T08:42:29.156" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-09-14T08:41:59.357" grpc.time_ms=123.125 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.789865] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 71 gitlab # [ 435.791937] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.554Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"031b241829bfecad51883231","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-031b241829bfecad51883231: done: 0.014228 sec","job_status":"done","queue_duration_s":2.03287,"scheduling_latency_s":2.032884,"redis_calls":13,"redis_duration_s":0.002654,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000922,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001732,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008824,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014228,"completed_at":"2026-09-14T08:41:57.554Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.831260] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21d484e878f66dbc6aefaf6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21d484e878f66dbc6aefaf6: start","job_status":"start","queue_duration_s":2.048419,"scheduling_latency_s":2.048433} gitlab # [ 435.844226] gitaly[897]: time="2026-09-14T08:41:59.552Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-09-14T08:42:29.350" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:59.550" grpc.time_ms=1.946 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.856379] gitlab-workhorse[820]: time="2026-09-14T08:41:59Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 435.858220] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:59 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 80 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses gitlab # [ 435.860166] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e21d484e878f66dbc6aefaf6","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e21d484e878f66dbc6aefaf6: done: 0.029784 sec","job_status":"done","queue_duration_s":2.048419,"scheduling_latency_s":2.048433,"redis_calls":13,"redis_duration_s":0.004788,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00158,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003208,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.029784,"completed_at":"2026-09-14T08:41:57.585Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.897845] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"018470c2942553d27c54eada","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-018470c2942553d27c54eada: start","job_status":"start","queue_duration_s":2.08179,"scheduling_latency_s":2.08183} gitlab # [ 435.909214] gitaly[897]: time="2026-09-14T08:41:59.585Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M2FH93ETSFCTYB3WTC0S9ZZ6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:59.574" grpc.time_ms=11.156 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 435.921226] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"018470c2942553d27c54eada","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-018470c2942553d27c54eada: done: 0.033024 sec","job_status":"done","queue_duration_s":2.08179,"scheduling_latency_s":2.08183,"redis_calls":13,"redis_duration_s":0.004115,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001429,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002686,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.033024,"completed_at":"2026-09-14T08:41:57.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.964654] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.624Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7867a1ff50e588d1d196497","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7867a1ff50e588d1d196497: start","job_status":"start","queue_duration_s":2.116848,"scheduling_latency_s":2.116872} gitlab # [ 435.976919] gitaly[897]: time="2026-09-14T08:41:59.602Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH92ENAFNKB72CRTSVY9TC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-09-14T08:42:29.400" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-09-14T08:41:59.600" grpc.time_ms=1.313 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.989400] gitaly[897]: time="2026-09-14T08:41:59.668Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M2FH93H49TW5Y5K8M29Q390Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:59.647" pid=897 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 435.999402] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c7867a1ff50e588d1d196497","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c7867a1ff50e588d1d196497: done: 0.024067 sec","job_status":"done","queue_duration_s":2.116848,"scheduling_latency_s":2.116872,"redis_calls":13,"redis_duration_s":0.004201,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003061,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015253,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024067,"completed_at":"2026-09-14T08:41:57.647Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.038708] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c774378c5990f564a3ca0c2","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c774378c5990f564a3ca0c2: start","job_status":"start","queue_duration_s":2.142741,"scheduling_latency_s":2.142763} gitlab # [ 436.051006] gitaly[897]: time="2026-09-14T08:41:59.668Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M2FH93H49TW5Y5K8M29Q390Z grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-09-14T08:41:59.647" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=897 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 436.064342] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/09/14:08:41:59 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 99 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/297480606dca023f7b934bf3a4c0ee99c9bd7e3a/statuses, in 0.20 seconds) (finished: subtest: eval and build post commit statuses, in 11.45 seconds) (finished: run the VM test script, in 508.16 seconds) test script finished in 508.19s cleanup kill QemuMachine (pid 46) gitlab # [ 436.066402] sidekiq-cluster[3414]: {"severity":"INFO","time":"2026-09-14T08:41:57.672Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8c774378c5990f564a3ca0c2","created_at":"2026-09-14T08:41:54.918Z","trace_propagation_headers":{"sentry-trace":"3dcd7c6c46554fbf937000e8ef753973-0505d3e7ad8f4f54","baggage":"sentry-trace_id=3dcd7c6c46554fbf937000e8ef753973,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"0cdb1d404e8dbedf0a10413ea9a79371","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-09-14T08:41:55.507Z","job_size_bytes":2,"pid":3414,"sidekiq_tid":"m8n2","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8c774378c5990f564a3ca0c2: done: 0.022826 sec","job_status":"done","queue_duration_s":2.142741,"scheduling_latency_s":2.142763,"redis_calls":13,"redis_duration_s":0.003424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00114,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002284,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014775,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022826,"completed_at":"2026-09-14T08:41:57.672Z","load_balancing_strategy":"primaqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) (finished: cleanup, in 1.60 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-09-14T08:40:24.081Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-09-14T08:41:48.036613Z', 'started_at': '2026-09-14T08:41:48.366408Z', 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': None}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': '2026-09-14T08:41:58.432Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-09-14T08:41:48.327Z', 'started_at': None, 'finished_at': '2026-09-14T08:41:58.432Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '297480606dca023f7b934bf3a4c0ee99c9bd7e3a', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-09-14T08:41:58.874Z', 'started_at': None, 'finished_at': '2026-09-14T08:41:59.068Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]