tribuchet: building on eliza Machine state will be reset. To keep it, pass --keep-machine-state start all VLans (finished: start all VLans, in 0.00 seconds) Test will time out and terminate in 3600 seconds run the VM test script start all VMs gitlab: starting vm gitlab: QEMU running (pid 46) gitlab # Disk image does not exist, creating the virtualisation disk image... gitlab # Formatting '/build/vm-state-gitlab/tmp.n5YyhlJLIW', fmt=raw size=1073741824 gitlab # mke2fs 1.47.4 (6-Mar-2025) (finished: start all VMs, in 72.57 seconds) gitlab # Discarding device blocks: 0/262144 done gitlab: waiting for unit setup-gitlab.service gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes gitlab: waiting for the VM to finish booting gitlab # Filesystem UUID: 14ec8fe1-2cb1-40fe-8412-34038da1f300 gitlab # Superblock backups stored on blocks: gitlab # 32768, 98304, 163840, 229376 gitlab # gitlab # Allocating group tables: 0/8 done gitlab # Writing inode tables: 0/8 done gitlab # Creating journal (8192 blocks): done gitlab # Writing superblocks and filesystem accounting information: 0/8 done gitlab # gitlab # Virtualisation disk image created. gitlab # Creating Nix store image... gitlab # Created Nix store image. gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40] gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 2026 gitlab # [ 0.000000] KASLR enabled gitlab # [ 0.000000] random: crng init done gitlab # [ 0.000000] Machine model: linux,dummy-virt gitlab # [ 0.000000] efi: UEFI not found. gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf] gitlab # [ 0.000000] Zone ranges: gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff] gitlab # [ 0.000000] DMA32 empty gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff] gitlab # [ 0.000000] Device empty gitlab # [ 0.000000] Movable zone start for each node gitlab # [ 0.000000] Early memory node ranges gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff] gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe000000 gitlab # [ 0.000000] psci: probing for conduit method from DT. gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware. gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs gitlab # [ 0.000000] psci: Trusted OS migration not required gitlab # [ 0.000000] psci: SMC Calling Convention v1.1 gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003) gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296 gitlab # [ 0.000000] Detected PIPT I-cache on CPU0 gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm) gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface gitlab # [ 0.000000] CPU features: detected: Spectre-v4 gitlab # [ 0.000000] CPU features: detected: Spectre-BHB gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38 gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23 gitlab # [ 0.000000] alternatives: applying boot alternatives gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/03m43jsmn0qbik050vwfbdyvsdsp251p-nixos-system-gitlab-test/init regInfo=/nix/store/m0jl9qzjzs37c1c8kz1v6dqqmcfbj81r-closure-info/registration console=ttyAMA0,115200n8 console=tty0 gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/m0jl9qzjzs37c1c8kz1v6dqqmcfbj81r-closure-info/registration", will be passed to user space. gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear) gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear) gitlab # [ 0.000000] software IO TLB: area num 4. gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB) gitlab # [ 0.000000] Fallback order for Node 0: 0 gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 1572864 gitlab # [ 0.000000] Policy zone: Normal gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 gitlab # [ 0.000000] allocated 12582912 bytes of page_ext gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups gitlab # [ 0.000000] rcu: Hierarchical RCU implementation. gitlab # [ 0.000000] rcu: RCU event tracing is enabled. gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4. gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled. gitlab # [ 0.000000] Rude variant of Tasks RCU enabled. gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled. gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies. gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4 gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4. gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0 gitlab # [ 0.000000] GICv3: 256 SPIs implemented gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0 gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000 gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff] gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1) gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f30000 gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f40000 gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt). gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns gitlab # [ 0.000030] arm-pv: using stolen time PV gitlab # [ 0.000374] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____) gitlab # [ 0.000577] Console: colour dummy device 80x25 gitlab # [ 0.000585] printk: legacy console [tty0] enabled gitlab # [ 0.000766] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000) gitlab # [ 0.000772] pid_max: default: 32768 minimum: 301 gitlab # [ 0.000860] LSM: initializing lsm=capability,landlock,yama,bpf,ima gitlab # [ 0.000983] landlock: Up and running. gitlab # [ 0.000986] Yama: becoming mindful. gitlab # [ 0.001397] LSM support for eBPF active gitlab # [ 0.001566] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.001678] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear) gitlab # [ 0.002893] cacheinfo: Unable to detect cache hierarchy for CPU 0 gitlab # [ 0.003610] rcu: Hierarchical SRCU implementation. gitlab # [ 0.003613] rcu: Max phase no-delay instances is 1000. gitlab # [ 0.003774] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level gitlab # [ 0.004808] fsl-mc MSI: its@8080000 domain created gitlab # [ 0.004902] EFI services will not be available. gitlab # [ 0.005107] smp: Bringing up secondary CPUs ... gitlab # [ 0.005749] Detected PIPT I-cache on CPU1 gitlab # [ 0.005844] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000 gitlab # [ 0.005966] GICv3: CPU1: using allocated LPI pending table @0x0000000100f50000 gitlab # [ 0.006088] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40] gitlab # [ 0.007216] Detected PIPT I-cache on CPU2 gitlab # [ 0.007324] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000 gitlab # [ 0.007448] GICv3: CPU2: using allocated LPI pending table @0x0000000100f60000 gitlab # [ 0.007575] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40] gitlab # [ 0.008692] Detected PIPT I-cache on CPU3 gitlab # [ 0.008807] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000 gitlab # [ 0.008929] GICv3: CPU3: using allocated LPI pending table @0x0000000100f70000 gitlab # [ 0.009052] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40] gitlab # [ 0.009546] smp: Brought up 1 node, 4 CPUs gitlab # [ 0.009573] SMP: Total of 4 processors activated. gitlab # [ 0.009576] CPU: All CPU(s) started at EL1 gitlab # [ 0.009585] CPU features: detected: Branch Target Identification gitlab # [ 0.009589] CPU features: detected: ARMv8.4 Translation Table Level gitlab # [ 0.009592] CPU features: detected: Instruction cache invalidation not required for I/D coherence gitlab # [ 0.009595] CPU features: detected: Data cache clean to the PoU not required for I/D coherence gitlab # [ 0.009599] CPU features: detected: Common not Private translations gitlab # [ 0.009602] CPU features: detected: CRC32 instructions gitlab # [ 0.009604] CPU features: detected: Data cache clean to Point of Deep Persistence gitlab # [ 0.009608] CPU features: detected: Data cache clean to Point of Persistence gitlab # [ 0.009611] CPU features: detected: Data independent timing control (DIT) gitlab # [ 0.009614] CPU features: detected: E0PD gitlab # [ 0.009616] CPU features: detected: Enhanced Counter Virtualization gitlab # [ 0.009619] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF) gitlab # [ 0.009622] CPU features: detected: Enhanced Virtualization Traps gitlab # [ 0.009625] CPU features: detected: Fine Grained Traps gitlab # [ 0.009628] CPU features: detected: Generic authentication (architected QARMA5 algorithm) gitlab # [ 0.009632] CPU features: detected: RCpc load-acquire (LDAPR) gitlab # [ 0.009635] CPU features: detected: LSE atomic instructions gitlab # [ 0.009637] CPU features: detected: Privileged Access Never gitlab # [ 0.009640] CPU features: detected: PMUv3 gitlab # [ 0.009642] CPU features: detected: RAS Extension Support gitlab # [ 0.009644] CPU features: detected: RASv1p1 Extension Support gitlab # [ 0.009647] CPU features: detected: Random Number Generator gitlab # [ 0.009649] CPU features: detected: Speculation barrier (SB) gitlab # [ 0.009652] CPU features: detected: Stage-2 Force Write-Back gitlab # [ 0.009654] CPU features: detected: TLB range maintenance instructions gitlab # [ 0.009658] CPU features: detected: Speculative Store Bypassing Safe (SSBS) gitlab # [ 0.009782] alternatives: applying system-wide alternatives gitlab # [ 0.012749] CPU features: detected: BBM Level 2 without TLB conflict abort gitlab # [ 0.012981] Memory: 5963792K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved) gitlab # [ 0.014467] devtmpfs: initialized gitlab # [ 0.018414] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear) gitlab # [ 0.018488] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear). gitlab # [ 0.018689] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL gitlab # [ 0.018694] 0 pages in range for non-PLT usage gitlab # [ 0.018695] 508288 pages in range for PLT usage gitlab # [ 0.018824] pinctrl core: initialized pinctrl subsystem gitlab # [ 0.019692] DMI not present or invalid. gitlab # [ 0.023283] NET: Registered PF_NETLINK/PF_ROUTE protocol family gitlab # [ 0.025693] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations gitlab # [ 0.025969] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations gitlab # [ 0.027717] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations gitlab # [ 0.027739] audit: initializing netlink subsys (disabled) gitlab # [ 0.028048] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1 gitlab # [ 0.028466] thermal_sys: Registered thermal governor 'fair_share' gitlab # [ 0.028469] thermal_sys: Registered thermal governor 'bang_bang' gitlab # [ 0.028473] thermal_sys: Registered thermal governor 'step_wise' gitlab # [ 0.028476] thermal_sys: Registered thermal governor 'user_space' gitlab # [ 0.028478] thermal_sys: Registered thermal governor 'power_allocator' gitlab # [ 0.028558] cpuidle: using governor ladder gitlab # [ 0.028617] cpuidle: using governor menu gitlab # [ 0.028822] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers. gitlab # [ 0.029036] ASID allocator initialised with 65536 entries gitlab # [ 0.030176] Serial: AMBA PL011 UART driver gitlab # [ 0.035388] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1 gitlab # [ 0.035690] printk: console [ttyAMA0] enabled gitlab # [ 0.042042] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages gitlab # [ 0.042065] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page gitlab # [ 0.042075] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages gitlab # [ 0.042083] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page gitlab # [ 0.042092] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages gitlab # [ 0.042100] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page gitlab # [ 0.042108] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages gitlab # [ 0.042116] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page gitlab # [ 0.055358] fbcon: Taking over console gitlab # [ 0.055394] ACPI: Interpreter disabled. gitlab # [ 0.058344] iommu: Default domain type: Translated gitlab # [ 0.058356] iommu: DMA domain TLB invalidation policy: strict mode gitlab # [ 0.065649] SCSI subsystem initialized gitlab # [ 0.066428] usbcore: registered new interface driver usbfs gitlab # [ 0.066529] usbcore: registered new interface driver hub gitlab # [ 0.066576] usbcore: registered new device driver usb gitlab # [ 0.067403] pps_core: LinuxPPS API ver. 1 registered gitlab # [ 0.067413] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti gitlab # [ 0.067437] PTP clock support registered gitlab # [ 0.067575] EDAC MC: Ver: 3.0.0 gitlab # [ 0.068230] scmi_core: SCMI protocol bus registered gitlab # [ 0.069550] FPGA manager framework gitlab # [ 0.072073] vgaarb: loaded gitlab # [ 0.074683] clocksource: Switched to clocksource arch_sys_counter gitlab # [ 0.077779] VFS: Disk quotas dquot_6.6.0 gitlab # [ 0.077848] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes) gitlab # [ 0.078887] netfs: FS-Cache loaded gitlab # [ 0.079422] pnp: PnP ACPI: disabled gitlab # [ 0.090727] NET: Registered PF_INET protocol family gitlab # [ 0.093289] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear) gitlab # [ 0.154777] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear) gitlab # [ 0.154855] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) gitlab # [ 0.154926] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear) gitlab # [ 0.155396] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear) gitlab # [ 0.156316] TCP: Hash tables configured (established 65536 bind 65536) gitlab # [ 0.156452] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear) gitlab # [ 0.156535] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.156709] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear) gitlab # [ 0.157051] NET: Registered PF_UNIX/PF_LOCAL protocol family gitlab # [ 0.157130] NET: Registered PF_XDP protocol family gitlab # [ 0.157158] PCI: CLS 0 bytes, default 64 gitlab # [ 0.157540] Trying to unpack rootfs image as initramfs... gitlab # [ 0.171360] kvm [1]: HYP mode not available gitlab # [ 0.248706] Initialise system trusted keyrings gitlab # [ 0.249173] workingset: timestamp_bits=42 max_order=21 bucket_order=0 gitlab # [ 0.250118] squashfs: version 4.0 (2009/01/31) Phillip Lougher gitlab # [ 0.250257] 9p: Installing v9fs 9p2000 file system support gitlab # [ 0.265577] Key type asymmetric registered gitlab # [ 0.265593] Asymmetric key parser 'x509' registered gitlab # [ 0.265664] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242) gitlab # [ 0.265991] io scheduler mq-deadline registered gitlab # [ 0.266008] io scheduler kyber registered gitlab # [ 0.280443] pl061_gpio 9030000.pl061: PL061 GPIO chip registered gitlab # [ 0.282891] ledtrig-cpu: registered to indicate activity on CPUs gitlab # [ 0.284124] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges: gitlab # [ 0.284178] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000 gitlab # [ 0.284226] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000 gitlab # [ 0.284245] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000 gitlab # [ 0.284310] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits gitlab # [ 0.284363] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff] gitlab # [ 0.284586] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00 gitlab # [ 0.284626] pci_bus 0000:00: root bus resource [bus 00-ff] gitlab # [ 0.284660] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] gitlab # [ 0.284672] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff] gitlab # [ 0.284682] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff] gitlab # [ 0.284955] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint gitlab # [ 0.286350] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.286994] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.287033] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.287113] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.287151] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.288633] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.289276] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.289311] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.289405] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.290942] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.291545] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.291593] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.291685] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.293220] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint gitlab # [ 0.293854] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.293889] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.293971] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.295567] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.296206] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.296242] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.296345] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.297836] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint gitlab # [ 0.298464] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f] gitlab # [ 0.298501] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.298603] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.300130] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint gitlab # [ 0.300796] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.300831] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.300911] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.300948] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref] gitlab # [ 0.302607] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint gitlab # [ 0.303258] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.303338] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.304875] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint gitlab # [ 0.305549] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.305629] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.307024] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint gitlab # [ 0.307658] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff] gitlab # [ 0.308498] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint gitlab # [ 0.309232] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f] gitlab # [ 0.309268] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.309350] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.310996] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint gitlab # [ 0.311651] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f] gitlab # [ 0.311685] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff] gitlab # [ 0.311774] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref] gitlab # [ 0.313695] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned gitlab # [ 0.313715] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned gitlab # [ 0.313726] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned gitlab # [ 0.313860] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned gitlab # [ 0.313990] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned gitlab # [ 0.314136] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned gitlab # [ 0.314273] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned gitlab # [ 0.314410] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned gitlab # [ 0.314544] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned gitlab # [ 0.314729] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned gitlab # [ 0.314888] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned gitlab # [ 0.315043] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned gitlab # [ 0.315210] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned gitlab # [ 0.315364] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned gitlab # [ 0.315418] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned gitlab # [ 0.315470] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned gitlab # [ 0.315524] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned gitlab # [ 0.315579] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned gitlab # [ 0.315646] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned gitlab # [ 0.315702] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned gitlab # [ 0.315756] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned gitlab # [ 0.315818] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned gitlab # [ 0.315874] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned gitlab # [ 0.315928] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned gitlab # [ 0.315987] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned gitlab # [ 0.316039] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned gitlab # [ 0.316092] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned gitlab # [ 0.316156] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned gitlab # [ 0.316210] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned gitlab # [ 0.316262] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned gitlab # [ 0.316316] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned gitlab # [ 0.316368] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned gitlab # [ 0.316436] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned gitlab # [ 0.316489] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned gitlab # [ 0.316570] pci_bus 0000:00: resource 4 [io 0x0000-0xffff] gitlab # [ 0.316586] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff] gitlab # [ 0.316595] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff] gitlab # [ 0.320648] pci 0000:00:0a.0: enabling device (0000 -> 0002) gitlab # [ 0.355636] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003) gitlab # [ 0.360700] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003) gitlab # [ 0.365773] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003) gitlab # [ 0.372081] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003) gitlab # [ 0.377284] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003) gitlab # [ 0.383296] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003) gitlab # [ 0.388115] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003) gitlab # [ 0.392872] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002) gitlab # [ 0.398899] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002) gitlab # [ 0.403381] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003) gitlab # [ 0.410019] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003) gitlab # [ 0.422659] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled gitlab # [ 0.426672] msm_serial: driver initialized gitlab # [ 0.427044] SuperH (H)SCI(F) driver initialized gitlab # [ 0.427159] STM32 USART driver initialized gitlab # [ 0.462511] loop: module loaded gitlab # [ 0.462961] virtio_blk virtio4: 4/0/0 default/read/poll queues gitlab # [ 0.466169] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB) gitlab # [ 0.469386] virtio_blk virtio5: 4/0/0 default/read/poll queues gitlab # [ 0.472667] virtio_blk virtio5: [vdb] 12227512 512-byte logical blocks (6.26 GB/5.83 GiB) gitlab # [ 0.476377] megasas: 07.734.00.00-rc1 gitlab # [ 0.477525] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff] gitlab # [ 0.482271] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.482362] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.485031] Using buffer write method gitlab # [ 0.485077] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff] gitlab # [ 0.488422] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000 gitlab # [ 0.488444] Intel/Sharp Extended Query Table at 0x0031 gitlab # [ 0.493222] Using buffer write method gitlab # [ 0.493247] Concatenating MTD devices: gitlab # [ 0.493251] (0): "0.flash" gitlab # [ 0.493255] (1): "0.flash" gitlab # [ 0.493258] into device "0.flash" gitlab # [ 0.631259] Freeing initrd memory: 27184K gitlab # [ 0.650394] tun: Universal TUN/TAP device driver, 1.6 gitlab # [ 0.658932] thunder_xcv, ver 1.0 gitlab # [ 0.659054] thunder_bgx, ver 1.0 gitlab # [ 0.659115] nicpf, ver 1.0 gitlab # [ 0.660876] e1000: Intel(R) PRO/1000 Network Driver gitlab # [ 0.660890] e1000: Copyright (c) 1999-2006 Intel Corporation. gitlab # [ 0.660958] e1000e: Intel(R) PRO/1000 Network Driver gitlab # [ 0.660973] e1000e: Copyright(c) 1999 - 2015 Intel Corporation. gitlab # [ 0.661065] igb: Intel(R) Gigabit Ethernet Network Driver gitlab # [ 0.661076] igb: Copyright (c) 2007-2014 Intel Corporation. gitlab # [ 0.661140] igbvf: Intel(R) Gigabit Virtual Function Network Driver gitlab # [ 0.661150] igbvf: Copyright (c) 2009 - 2012 Intel Corporation. gitlab # [ 0.661604] sky2: driver version 1.30 gitlab # [ 0.666711] usbcore: registered new interface driver usb-storage gitlab # [ 0.666963] ehci-pci 0000:00:0a.0: EHCI Host Controller gitlab # [ 0.667012] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1 gitlab # [ 0.667155] usbcore: registered new interface driver usbserial_generic gitlab # [ 0.667204] usbserial: USB Serial support registered for generic gitlab # [ 0.667739] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000 gitlab # [ 0.669108] hv_vmbus: registering driver hyperv_keyboard gitlab # [ 0.671808] rtc-pl031 9010000.pl031: registered as rtc0 gitlab # [ 0.671868] rtc-pl031 9010000.pl031: setting system clock to 2026-08-15T13:20:28 UTC (1786800028) gitlab # [ 0.672801] i2c_dev: i2c /dev entries driver gitlab # [ 0.681428] sdhci: Secure Digital Host Controller Interface driver gitlab # [ 0.681442] sdhci: Copyright(c) Pierre Ossman gitlab # [ 0.682262] Synopsys Designware Multimedia Card Interface Driver gitlab # [ 0.682787] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00 gitlab # [ 0.683475] sdhci-pltfm: SDHCI platform and OF driver helper gitlab # [ 0.683962] hub 1-0:1.0: USB hub found gitlab # [ 0.684065] hub 1-0:1.0: 6 ports detected gitlab # [ 0.688569] hid: raw HID events driver (C) Jiri Kosina gitlab # [ 0.689436] usbcore: registered new interface driver usbhid gitlab # [ 0.689447] usbhid: USB HID core driver gitlab # [ 0.716744] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available gitlab # [ 0.721598] drop_monitor: Initializing network drop monitor service gitlab # [ 0.721920] NET: Registered PF_INET6 protocol family gitlab # [ 0.724065] Segment Routing with IPv6 gitlab # [ 0.724108] In-situ OAM (IOAM) with IPv6 gitlab # [ 0.724264] NET: Registered PF_PACKET protocol family gitlab # [ 0.724702] 9pnet: Installing 9P2000 support gitlab # [ 0.729130] Key type dns_resolver registered gitlab # [ 0.748194] registered taskstats version 1 gitlab # [ 0.748737] Loading compiled-in X.509 certificates gitlab # [ 0.770454] Demotion targets for Node 0: null gitlab # [ 0.770705] Key type .fscrypt registered gitlab # [ 0.770712] Key type fscrypt-provisioning registered gitlab # [ 0.770915] ima: No TPM chip found, activating TPM-bypass! gitlab # [ 0.770938] ima: Allocated hash algorithm: sha1 gitlab # [ 0.770968] ima: No architecture policies found gitlab # [ 0.772236] input: gpio-keys as /devices/platform/gpio-keys/input/input0 gitlab # [ 0.796559] clk: Disabling unused clocks gitlab # [ 0.796599] PM: genpd: Disabling unused power domains gitlab # [ 0.801359] Freeing unused kernel memory: 4736K gitlab # [ 0.801593] Run /init as init process gitlab # [ 0.828575] systemd[1]: Successfully made /usr/ read-only. gitlab # [ 0.930776] usb 1-1: new high-speed USB device number 2 using ehci-pci gitlab # [ 1.083416] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1 gitlab # [ 1.163368] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 1.163450] systemd[1]: Detected virtualization qemu. gitlab # [ 1.163547] systemd[1]: Detected architecture arm64. gitlab # [ 1.163566] systemd[1]: Running in initrd. gitlab # [ 1.164596] systemd[1]: Initializing machine ID from random generator. gitlab # [ 1.164995] systemd[1]: Hostname set to . gitlab # [ 1.199317] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0 gitlab # [ 1.318795] usb 1-2: new high-speed USB device number 3 using ehci-pci gitlab # [ 1.409370] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 1.474966] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2 gitlab # [ 1.475316] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0 gitlab # [ 1.518917] systemd[1]: Queued start job for default target Initrd Default Target. gitlab # [ 1.544542] systemd[1]: Created slice Slice /system/modprobe. gitlab # [ 1.545055] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 1.545130] systemd[1]: Expecting device /dev/disk/by-label/nix-store... gitlab # [ 1.545160] systemd[1]: Expecting device /dev/disk/by-label/nixos... gitlab # [ 1.545209] systemd[1]: Reached target Path Units. gitlab # [ 1.545247] systemd[1]: Reached target Slice Units. gitlab # [ 1.545288] systemd[1]: Reached target Swaps. gitlab # [ 1.545325] systemd[1]: Reached target Timer Units. gitlab # [ 1.545680] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 1.546025] systemd[1]: Listening on Journal Socket (/dev/log). gitlab # [ 1.546351] systemd[1]: Listening on Journal Sockets. gitlab # [ 1.546662] systemd[1]: Listening on udev Control Socket. gitlab # [ 1.546905] systemd[1]: Listening on udev Kernel Socket. gitlab # [ 1.546952] systemd[1]: Reached target Socket Units. gitlab # [ 1.550295] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 1.553811] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 1.553956] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 1.557682] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 1.581808] systemd[1]: Starting Journal Service... gitlab # [ 1.590033] systemd[1]: Starting Load Kernel Modules... gitlab # [ 1.590371] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 1.596152] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 1.603466] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 1.606494] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 1.607214] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 1.607947] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 1.610590] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 1.635025] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0 gitlab # [ 1.635299] [drm] features: -virgl +edid -resource_blob -host_visible gitlab # [ 1.635316] [drm] features: -context_init gitlab # [ 1.636836] [drm] number of scanouts: 1 gitlab # [ 1.636861] [drm] number of cap sets: 0 gitlab # [ 1.643281] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic gitlab # [ 1.643313] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0 gitlab # [ 1.644549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log. gitlab # [ 1.645374] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 1.650789] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev gitlab # [ 1.652257] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 1.659561] Console: switching to colour frame buffer device 160x50 gitlab # [ 1.661889] systemd-journald[94]: Collecting audit messages is disabled. gitlab # [ 1.666420] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device gitlab # [ 1.677435] systemd[1]: Finished Load Kernel Modules. gitlab # [ 1.686942] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 1.691083] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 1.693240] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 1.695075] systemd[1]: Reached target Local File Systems. gitlab # [ 1.719027] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 1.728505] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 1.750250] systemd[1]: Started Journal Service. gitlab # [ 1.747448] systemd-modules-load[95]: Using 4 probe threads gitlab # [ 1.750080] systemd-modules-load[95]: Module 'virtio_balloon' is built in gitlab # [ 1.752363] systemd-modules-load[95]: Module 'virtio_console' is built in gitlab # [ 1.753575] systemd-modules-load[95]: Module 'virtio_rng' is built in gitlab # [ 1.754671] systemd-modules-load[95]: Inserted module 'dm_mod' gitlab # [ 1.755803] systemd-modules-load[95]: Inserted module 'virtio_gpu' gitlab # [ 1.756930] systemd[1]: Starting Create System Files and Directories... gitlab # [ 1.767280] systemd-udevd[107]: Using default interface naming scheme 'v261'. gitlab # [ 1.786405] systemd[1]: Finished Create System Files and Directories. gitlab # [ 1.788320] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 1.807463] systemd[1]: Starting Virtual Console Setup... gitlab # [ 1.856149] systemd-vconsole-setup[130]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 1.859502] systemd[1]: Finished Virtual Console Setup. gitlab # [ 2.194974] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 2.197652] systemd[1]: Reached target System Initialization. gitlab # [ 2.200169] systemd[1]: Reached target Basic System. gitlab # [ 2.356339] (udev-worker)[137]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 2.358938] (udev-worker)[137]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.390807] (udev-worker)[135]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 2.404123] systemd[1]: Found device /dev/disk/by-label/nix-store. gitlab # [ 2.409762] systemd[1]: Found device /dev/disk/by-label/nixos. gitlab # [ 2.410789] systemd[1]: Reached target Initrd Root Device. gitlab # [ 2.412296] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos... gitlab # [ 2.459417] systemd-fsck[152]: nixos: clean, 12/65536 files, 13019/262144 blocks gitlab # [ 2.464034] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos. gitlab # [ 2.598100] systemd[1]: Mounting /sysroot... gitlab # [ 2.662327] EXT4-fs (vda): mounted filesystem 14ec8fe1-2cb1-40fe-8412-34038da1f300 r/w with ordered data mode. Quota mode: none. gitlab # [ 2.663206] systemd[1]: Mounted /sysroot. gitlab # [ 2.664581] systemd[1]: Reached target Initrd Root File System. gitlab # [ 2.669295] systemd[1]: Mounting /sysroot/nix/.ro-store... gitlab # [ 2.672420] systemd[1]: Mounting /sysroot/nix/.rw-store... gitlab # [ 2.674606] systemd[1]: Mounting /sysroot/run... gitlab # [ 2.696235] systemd[1]: Mounting /sysroot/tmp/shared... gitlab # [ 2.697918] systemd[1]: Mounting /sysroot/tmp/xchg... gitlab # [ 2.704328] systemd[1]: Starting Mountpoints Configured in the Real Root... gitlab # [ 2.709331] systemd[1]: Mounted /sysroot/nix/.rw-store. gitlab # [ 2.710593] systemd[1]: Mounted /sysroot/run. gitlab # [ 2.712095] systemd[1]: Mounted /sysroot/tmp/shared. gitlab # [ 2.715767] systemd-sysroot-fstab-check[165]: /sysroot should be mounted in the initrd, will request daemon-reload. gitlab # [ 2.719483] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 2.727727] systemd[1]: Reload requested from client PID 165 ('systemd-sysroot') (unit initrd-parse-etc.service)... gitlab # [ 2.730222] systemd[1]: Reloading... gitlab # [ 2.998780] erofs (device vdb): mounted with root inode @ nid 128. gitlab # [ 3.028109] (udev-worker)[123]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory gitlab # [ 3.031225] (udev-worker)[123]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory gitlab # [ 3.040895] systemd[1]: Reloading finished in 312 ms. gitlab # [ 3.075005] systemd-sysroot-fstab-check[165]: Requesting initrd-fs.target/start/replace... gitlab # [ 3.078172] systemd[1]: Mounted /sysroot/nix/.ro-store. gitlab # [ 3.079527] systemd[1]: Mounted /sysroot/tmp/xchg. gitlab # [ 3.080875] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.082393] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.087437] systemd-sysroot-fstab-check[165]: Requesting swap.target/start/replace... gitlab # [ 3.094111] systemd[1]: initrd-parse-etc.service: Deactivated successfully. gitlab # [ 3.096345] systemd[1]: Finished Mountpoints Configured in the Real Root. gitlab # [ 3.098312] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies. gitlab # [ 3.107175] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.109021] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.111101] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.112651] systemd[1]: Stopping Virtual Console Setup... gitlab # [ 3.114202] systemd[1]: Starting Virtual Console Setup... gitlab # [ 3.140554] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.144433] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.156378] systemd-vconsole-setup[198]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 3.161664] systemd[1]: Finished Virtual Console Setup. gitlab # [ 3.606622] systemd[1]: Mounting /sysroot/nix/store... gitlab # [ 3.699520] systemd[1]: Mounted /sysroot/nix/store. gitlab # [ 3.702014] systemd[1]: Reached target Initrd File Systems. gitlab # [ 3.705155] systemd[1]: Starting Find NixOS closure... gitlab # [ 3.709236] systemd[1]: Starting Create Volatile Files and Directories in the Real Root... gitlab # [ 3.766637] systemd[1]: Finished Create Volatile Files and Directories in the Real Root. gitlab # [ 3.771277] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully. gitlab # [ 3.785306] systemd[1]: Finished Find NixOS closure. gitlab # [ 3.787531] systemd[1]: Reached target Initrd Default Target. gitlab # [ 3.790170] systemd[1]: Starting Cleaning Up and Shutting Down Daemons... gitlab # [ 3.849890] systemd[1]: Stopped target Initrd Default Target. gitlab # [ 3.852605] systemd[1]: Stopped target Basic System. gitlab # [ 3.854810] systemd[1]: Stopped target Initrd Root Device. gitlab # [ 3.857358] systemd[1]: Stopped target Path Units. gitlab # [ 3.859506] systemd[1]: systemd-ask-password-console.path: Deactivated successfully. gitlab # [ 3.862920] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch. gitlab # [ 3.866410] systemd[1]: Stopped target Slice Units. gitlab # [ 3.868816] systemd[1]: Stopped target Socket Units. gitlab # [ 3.871200] systemd[1]: Stopped target System Initialization. gitlab # [ 3.874326] systemd[1]: Stopped target Swaps. gitlab # [ 3.876887] systemd[1]: Stopped target Timer Units. gitlab # [ 3.881394] systemd[1]: dbus.socket: Deactivated successfully. gitlab # [ 3.883946] systemd[1]: Closed D-Bus System Message Bus Socket. gitlab # [ 3.888268] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully. gitlab # [ 3.893814] systemd[1]: Stopped Find NixOS closure. gitlab # [ 3.896285] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 3.899328] systemd[1]: Starting rw-sysroot-nix-store.service... gitlab # [ 3.904730] systemd[1]: systemd-sysctl.service: Deactivated successfully. gitlab # [ 3.907201] systemd[1]: Stopped Apply Kernel Variables. gitlab # [ 3.912571] systemd[1]: systemd-modules-load.service: Deactivated successfully. gitlab # [ 3.915565] systemd[1]: Stopped Load Kernel Modules. gitlab # [ 3.917752] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully. gitlab # [ 3.920687] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root. gitlab # [ 3.923506] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully. gitlab # [ 3.926195] systemd[1]: Stopped Create System Files and Directories. gitlab # [ 3.928362] systemd[1]: Stopped target Local File Systems. gitlab # [ 3.930238] systemd[1]: Stopped target Preparation for Local File Systems. gitlab # [ 3.932872] systemd[1]: systemd-udev-trigger.service: Deactivated successfully. gitlab # [ 3.935027] systemd[1]: Stopped Coldplug All udev Devices. gitlab # [ 3.936996] systemd[1]: Stopping Rule-based Manager for Device Events and Files... gitlab # [ 3.939180] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully. gitlab # [ 3.941430] systemd[1]: Stopped Virtual Console Setup. gitlab # [ 3.942888] systemd[1]: initrd-cleanup.service: Deactivated successfully. gitlab # [ 3.944980] systemd[1]: Finished Cleaning Up and Shutting Down Daemons. gitlab # [ 3.946832] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 3.948954] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 3.950781] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully. gitlab # [ 3.952830] systemd[1]: Finished rw-sysroot-nix-store.service. gitlab # [ 3.954508] systemd[1]: systemd-udevd.service: Deactivated successfully. gitlab # [ 3.956363] systemd[1]: Stopped Rule-based Manager for Device Events and Files. gitlab # [ 3.958349] systemd[1]: systemd-udevd.service: Consumed 2.339s CPU time over 2.222s wall clock time, 31.4M memory peak. gitlab # [ 3.961182] systemd[1]: systemd-udevd-control.socket: Deactivated successfully. gitlab # [ 3.963231] systemd[1]: Closed udev Control Socket. gitlab # [ 3.964838] systemd[1]: Starting Cleanup udev Database... gitlab # [ 3.966441] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully. gitlab # [ 3.968611] systemd[1]: Stopped Create Static Device Nodes in /dev. gitlab # [ 3.970342] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully. gitlab # [ 3.972772] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully. gitlab # [ 3.974999] systemd[1]: kmod-static-nodes.service: Deactivated successfully. gitlab # [ 3.977225] systemd[1]: Stopped Create List of Static Device Nodes. gitlab # [ 4.084135] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully. gitlab # [ 4.087622] systemd[1]: Finished Cleanup udev Database. gitlab # [ 4.090221] systemd[1]: Reached target Switch Root. gitlab # [ 4.092439] systemd[1]: Starting NixOS Activation... gitlab # [ 4.333698] initrd-nixos-activation-start[225]: booting system configuration /nix/store/03m43jsmn0qbik050vwfbdyvsdsp251p-nixos-system-gitlab-test gitlab # [ 4.428164] initrd-nixos-activation-start[225]: running activation script... gitlab # [ 4.905382] initrd-nixos-activation-start[248]: setting up /etc... gitlab # [ 5.139423] systemd[1]: initrd-nixos-activation.service: Deactivated successfully. gitlab # [ 5.142046] systemd[1]: Finished NixOS Activation. gitlab # [ 5.148444] systemd[1]: Starting Switch Root... gitlab # [ 5.221180] systemd[1]: Switching root. gitlab # [ 5.386184] systemd-journald[94]: Received SIGTERM from PID 1 (systemd). gitlab # [ 6.049031] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE) gitlab # [ 6.049173] systemd[1]: Detected virtualization qemu. gitlab # [ 6.049277] systemd[1]: Detected architecture arm64. gitlab # [ 6.049459] systemd[1]: Detected first boot. gitlab # [ 6.050954] systemd[1]: Initializing machine ID from random generator. gitlab # [ 6.289751] systemd[1]: bpf-restrict-fs: LSM BPF program attached gitlab # [ 6.431518] systemd[1]: Applying preset policy. gitlab # [ 6.674343] systemd[1]: Populated /etc with preset unit settings. gitlab # [ 6.928188] systemd[1]: initrd-switch-root.service: Deactivated successfully. gitlab # [ 6.928975] systemd[1]: Stopped initrd-switch-root.service. gitlab # [ 6.934318] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1. gitlab # [ 6.940738] systemd[1]: Created slice Slice /system/getty. gitlab # [ 6.944768] systemd[1]: Created slice GitLab DevOps Platform Slice. gitlab # [ 6.948658] systemd[1]: Created slice User and Session Slice. gitlab # [ 6.949538] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. gitlab # [ 6.950555] systemd[1]: Started Forward Password Requests to Wall Directory Watch. gitlab # [ 6.951517] systemd[1]: Expecting device /dev/hvc0... gitlab # [ 6.952346] systemd[1]: Expecting device /dev/ttyAMA0... gitlab # [ 6.953276] systemd[1]: Reached target Local Encrypted Volumes. gitlab # [ 6.954189] systemd[1]: Stopped target initrd-fs.target. gitlab # [ 6.955096] systemd[1]: Stopped target initrd-root-fs.target. gitlab # [ 6.956014] systemd[1]: Stopped target initrd-switch-root.target. gitlab # [ 6.956924] systemd[1]: Reached target Virtual Machines and Containers. gitlab # [ 6.957814] systemd[1]: Reached target Path Units. gitlab # [ 6.958728] systemd[1]: Reached target Remote File Systems. gitlab # [ 6.959572] systemd[1]: Reached target Slice Units. gitlab # [ 6.960491] systemd[1]: Reached target Swaps. gitlab # [ 6.963258] systemd[1]: Listening on Query the User Interactively for a Password. gitlab # [ 6.965940] systemd[1]: Listening on Process Core Dump Socket. gitlab # [ 6.967890] systemd[1]: Listening on Credential Encryption/Decryption. gitlab # [ 6.969967] systemd[1]: Listening on Factory Reset Management. gitlab # [ 6.970847] systemd[1]: Listening on Hostname Service Socket. gitlab # [ 6.976412] systemd[1]: Starting Journal Log Access Socket... gitlab # [ 6.978000] systemd[1]: Listening on Journal Audit Socket. gitlab # [ 6.980498] systemd[1]: Listening on Console Output Muting Service Socket. gitlab # [ 6.981516] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket. gitlab # [ 6.982224] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 6.983093] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki gitlab # [ 6.986875] systemd[1]: Listening on Disk Repartitioning Service Socket. gitlab # [ 6.987713] systemd[1]: Listening on udev Control Socket. gitlab # [ 6.988727] systemd[1]: Listening on udev Varlink Socket. gitlab # [ 7.052326] systemd[1]: Mounting Huge Pages File System... gitlab # [ 7.058188] systemd[1]: Mounting POSIX Message Queue File System... gitlab # [ 7.065167] systemd[1]: Mounting Kernel Debug File System... gitlab # [ 7.073610] systemd[1]: Mounting Kernel Trace File System... gitlab # [ 7.081601] systemd[1]: Starting Create List of Static Device Nodes... gitlab # [ 7.090871] systemd[1]: Starting Load Kernel Module 9pnet_virtio... gitlab # [ 7.092222] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.111777] systemd[1]: Mounting Kernel Configuration File System... gitlab # [ 7.115184] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm gitlab # [ 7.116171] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 7.130970] systemd[1]: Starting Load Kernel Module fuse... gitlab # [ 7.135231] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 7.153687] systemd[1]: Starting Journal Service... gitlab # [ 7.163056] systemd[1]: Starting Load Kernel Modules... gitlab # [ 7.175525] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer... gitlab # [ 7.186644] fuse: init (API version 7.45) gitlab # [ 7.186936] systemd[1]: Starting Remount Root and Kernel File Systems... gitlab # [ 7.187962] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.197945] systemd[1]: Starting Coldplug All udev Devices... gitlab # [ 7.203403] systemd[1]: Listening on Journal Log Access Socket. gitlab # [ 7.204242] systemd[1]: Mounted Huge Pages File System. gitlab # [ 7.205162] systemd[1]: Mounted POSIX Message Queue File System. gitlab # [ 7.206058] systemd[1]: Mounted Kernel Debug File System. gitlab # [ 7.206989] systemd[1]: Mounted Kernel Trace File System. gitlab # [ 7.210280] systemd[1]: Finished Create List of Static Device Nodes. gitlab # [ 7.211343] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully. gitlab # [ 7.212194] systemd[1]: Finished Load Kernel Module 9pnet_virtio. gitlab # [ 7.212981] systemd[1]: Mounted Kernel Configuration File System. gitlab # [ 7.213928] systemd[1]: modprobe@fuse.service: Deactivated successfully. gitlab # [ 7.214937] systemd[1]: Finished Load Kernel Module fuse. gitlab # [ 7.222168] systemd-journald[319]: Collecting audit messages is enabled. gitlab # [ 7.223055] systemd[1]: Mounting FUSE Control File System... gitlab # [ 7.235956] systemd[1]: Starting Create Static Device Nodes in /dev gracefully... gitlab # [ 7.241323] systemd[1]: Finished Load Kernel Modules. gitlab # [ 7.245722] systemd[1]: Queued start job for default target Multi-User System. gitlab # [ 7.247351] systemd[1]: Unnecessary job was removed for /dev/vdb. gitlab # [ 7.252654] systemd[1]: systemd-journald.service: Deactivated successfully.[ 7.254989] systemd[1]: Starting Firewall... gitlab # gitlab # [ 7.255826] systemd-modules-load[320]: Using 3 probe threads gitlab # [ 7.257080] systemd-modules-load[320]: Module 'atkbd' is built in gitlab # [ 7.258407] systemd-modules-load[320]: Module 'loop' is built in gitlab # [ 7.264478] systemd[1]: Starting Apply Kernel Variables... gitlab # [ 7.264381] systemd-modules-load[320]: Inserted module 'tls'[ 7.267218] EXT4-fs (vda): re-mounted 14ec8fe1-2cb1-40fe-8412-34038da1f300. gitlab # gitlab # [ 7.272462] systemd[1]: Started Journal Service. gitlab # [ 7.278868] systemd[1]: Finished Remount Root and Kernel File Systems. gitlab # [ 7.280590] systemd[1]: Mounted FUSE Control File System. gitlab # [ 7.286708] systemd[1]: Listening on Disk Image Download Service Socket. gitlab # [ 7.291093] systemd-oomd[321]: No swap; memory pressure usage will be degraded gitlab # [ 7.293766] systemd[1]: Starting Flush Journal to Persistent Storage... gitlab # [ 7.300341] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 7.303926] systemd[1]: Starting Load/Save OS Random Seed... gitlab # [ 7.305319] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 7.307138] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer. gitlab # [ 7.320679] systemd[1]: Finished Create Static Device Nodes in /dev gracefully. gitlab # [ 7.327236] systemd[1]: Finished Apply Kernel Variables. gitlab # [ 7.332561] systemd[1]: Starting Create Static Device Nodes in /dev... gitlab # [ 7.341913] systemd-journald[319]: Received client request to flush runtime journal. gitlab # [ 7.377194] systemd[1]: Finished Load/Save OS Random Seed. gitlab # [ 7.378313] systemd[1]: Reached target First Boot Complete. gitlab # [ 7.379246] systemd[1]: Finished Create Static Device Nodes in /dev. gitlab # [ 7.381566] systemd[1]: Reached target Preparation for Local File Systems. gitlab # [ 7.382846] systemd[1]: Starting Rule-based Manager for Device Events and Files... gitlab # [ 7.384444] systemd[1]: Finished Flush Journal to Persistent Storage. gitlab # [ 7.448939] systemd-udevd[363]: Using default interface naming scheme 'v261'. gitlab # [ 7.507688] systemd[1]: Started Rule-based Manager for Device Events and Files. gitlab # [ 7.753615] systemd[1]: Finished Firewall. gitlab # [ 7.865960] systemd[1]: Finished Coldplug All udev Devices. gitlab # [ 7.938502] systemd[1]: Mounting /run/wrappers... gitlab # [ 7.944484] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs gitlab # [ 7.993901] systemd[1]: Mounted /run/wrappers. gitlab # [ 8.000412] systemd[1]: Reached target Local File Systems. gitlab # [ 8.006748] systemd[1]: Listening on Boot Loader Control Service Socket. gitlab # [ 8.011956] systemd[1]: Starting register-nix-paths.service... gitlab # [ 8.021873] systemd[1]: Starting Create SUID/SGID Wrappers... gitlab # [ 8.023859] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 8.028470] systemd[1]: Starting Save Transient machine-id to Disk... gitlab # [ 8.048279] systemd[1]: Starting Create System Files and Directories... gitlab # [ 8.098982] systemd[1]: etc-machine\x2did.mount: Deactivated successfully. gitlab # [ 8.100583] systemd[1]: Finished Save Transient machine-id to Disk. gitlab # [ 8.103760] systemd[1]: Finished Create System Files and Directories. gitlab # [ 8.109278] systemd[1]: Condition check resulted in /dev/hvc0 being skipped. gitlab # [ 8.116140] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped. gitlab # [ 8.120916] (udev-worker)[424]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 8.122750] (udev-worker)[425]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name. gitlab # [ 8.125774] (udev-worker)[425]: Network interface NamePolicy= disabled on kernel command line. gitlab # [ 8.130447] systemd[1]: Starting Rebuild Journal Catalog... gitlab # [ 8.139987] systemd[1]: Starting Record System Boot/Shutdown in UTMP... gitlab # [ 8.145809] systemd[1]: Condition check resulted in Virtio network device being skipped. gitlab # [ 8.147234] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore gitlab # [ 8.149156] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met. gitlab # [ 8.151172] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67 gitlab # [ 8.154824] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore gitlab # [ 8.159145] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 8.163153] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os gitlab # [ 8.166665] systemd[1]: Finished Record System Boot/Shutdown in UTMP. gitlab # [ 8.191597] mousedev: PS/2 mouse device common for all mice gitlab # [ 8.194138] systemd[1]: Finished Rebuild Journal Catalog. gitlab # [ 8.196361] systemd[1]: Starting Update is Completed... gitlab # [ 8.231483] systemd[1]: Finished Update is Completed. gitlab # [ 8.247278] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully. gitlab # [ 8.248614] systemd[1]: Finished Create SUID/SGID Wrappers. gitlab # [ 8.249513] systemd[1]: suid-sgid-wrappers.service: Consumed 145ms CPU time over 227ms wall clock time, 6.2M memory peak, 1.9M read from disk. gitlab # [ 8.572501] systemd[1]: Finished register-nix-paths.service. gitlab # [ 8.573575] systemd[1]: Reached target System Initialization. gitlab # [ 8.574634] systemd[1]: Started Discard unused filesystem blocks once a week. gitlab # [ 8.575685] systemd[1]: Started logrotate.timer. gitlab # [ 8.576801] systemd[1]: Started Daily Cleanup of Temporary Directories. gitlab # [ 8.577766] systemd[1]: Reached target Timer Units. gitlab # [ 8.578516] systemd[1]: Listening on D-Bus System Message Bus Socket. gitlab # [ 8.579429] systemd[1]: Listening on Nix Daemon Socket. gitlab # [ 8.600111] systemd[1]: Starting nixbot CI service socket... gitlab # [ 8.601126] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket. gitlab # [ 8.603986] systemd[1]: Starting D-Bus System Message Bus... gitlab # [ 8.605292] systemd[1]: Listening on nixbot CI service socket. gitlab # [ 8.606344] systemd[1]: Reached target Socket Units. gitlab # [ 8.653268] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3 gitlab # [ 8.651443] dbus-broker-launch[566]: Looking up NSS user entry for 'systemd-timesync'... gitlab # [ 8.655098] dbus-broker-launch[566]: NSS returned no entry for 'systemd-timesync' gitlab # [ 8.656625] dbus-broker-launch[566]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync" gitlab # [ 8.663913] systemd[1]: Started D-Bus System Message Bus. gitlab # [ 8.676647] dbus-broker-launch[566]: Ready gitlab # [ 8.677515] systemd[1]: Reached target Basic System. gitlab # [ 8.679277] systemd[1]: Started backdoor.service. gitlab # [ 8.684766] systemd[1]: Starting gitlab-config.service... gitlab # [ 8.689427] systemd[1]: Starting Import lastlog data into lastlog2 database... gitlab # [ 8.694522] systemd[1]: Starting Logrotate configuration check... gitlab # [ 8.696670] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # connecting to host... gitlab # [ 8.724147] systemd[1]: Starting Post-Boot Actions... gitlab # [ 8.725028] systemd[1]: Started Reset console on configuration changes. gitlab # [ 8.730308] logrotate[571]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want. gitlab # [ 8.732506] logrotate[571]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf gitlab # [ 8.733895] logrotate[571]: note: 'monthly' overrides previously specified 'weekly' gitlab # [ 8.734961] logrotate[571]: Reading state from file: /var/lib/logrotate.status gitlab # [ 8.737631] logrotate[571]: state file /var/lib/logrotate.status does not exist gitlab # [ 8.743925] logrotate[571]: Allocating hash table for state file, size 64 entries gitlab # [ 8.745266] logrotate[571]: Handling 4 logs gitlab # [ 8.745862] logrotate[571]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.747894] logrotate[571]: considering log /var/log/btmp gitlab # [ 8.749237] logrotate[571]: Creating new state gitlab # [ 8.750904] logrotate[571]: Now: 2026-08-15 13:20 gitlab # [ 8.751623] logrotate[571]: Last rotated at 2026-08-15 13:00 gitlab # [ 8.752471] logrotate[571]: log does not need rotating (log has already been rotated) gitlab # [ 8.753573] logrotate[571]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed gitlab # [ 8.755777] logrotate[571]: considering log /var/log/wtmp gitlab # [ 8.756582] logrotate[571]: Creating new state gitlab # [ 8.757237] logrotate[571]: Now: 2026-08-15 13:20 gitlab # [ 8.757955] logrotate[571]: Last rotated at 2026-08-15 13:00 gitlab: Guest shell says: b'Spawning backdoor root shell...\n' gitlab # [ 8.760583] logrotate[571]: log does not need rotating (log has already been rotated) gitlab # [ 8.761625] logrotate[571]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed gitlab # [ 8.763218] logrotate[571]: switching euid from 0 to 165 and egid from 0 to 165 (pid 571) gitlab # [ 8.764292] logrotate[571]: considering log /var/gitlab/state/log/*.log gitlab: connected to guest root shell gitlab # [ 8.765092] logrotate[571]: log /var/gitlab/state/log/*.log does not exist -- skipping gitlab: (connecting took 9.18 seconds) gitlab # [ 8.766031] logrotate[571]: Creating new state gitlab: (finished: waiting for the VM to finish booting, in 9.18 seconds) gitlab # [ 8.766582] logrotate[571]: switching euid from 165 to 0 and egid from 165 to 0 (pid 571) gitlab # [ 8.767592] logrotate[571]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed gitlab # [ 8.769172] logrotate[571]: switching euid from 0 to 60 and egid from 0 to 60 (pid 571) gitlab # [ 8.770101] logrotate[571]: considering log /var/log/nginx/*.log gitlab # [ 8.770814] logrotate[571]: log /var/log/nginx/*.log does not exist -- skipping gitlab # [ 8.771683] logrotate[571]: Creating new state gitlab # [ 8.772258] logrotate[571]: not running postrotate script, since no logs were rotated gitlab # [ 8.773228] logrotate[571]: switching euid from 60 to 0 and egid from 60 to 0 (pid 571) gitlab # [ 8.774584] nsncd[572]: Aug 15 13:20:36.590 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.776613] systemd[1]: Starting resolvconf update... gitlab # [ 8.777412] systemd[1]: Finished Logrotate configuration check. gitlab # [ 8.778295] systemd[1]: Finished Post-Boot Actions. gitlab # [ 8.779061] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.779989] systemd[1]: Finished Import lastlog data into lastlog2 database. gitlab # [ 8.781323] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.782215] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.783101] systemd[1]: Starting User Login Management... gitlab # [ 8.783937] systemd[1]: Starting Virtual Console Setup... gitlab # [ 8.834797] systemd-logind[616]: New seat seat0. gitlab # [ 8.837731] systemd-logind[616]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard) gitlab # [ 8.840886] systemd-logind[616]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard) gitlab # [ 8.842610] systemd-logind[616]: Watching system buttons on /dev/input/event0 (gpio-keys) gitlab # [ 8.843689] systemd[1]: Started User Login Management. gitlab # [ 8.845326] systemd[1]: Stopped target Host and Network Name Lookups. gitlab # [ 8.846242] systemd[1]: Stopping Host and Network Name Lookups... gitlab # [ 8.847065] systemd[1]: Stopped target User and Group Name Lookups. gitlab # [ 8.847905] systemd[1]: Stopping User and Group Name Lookups... gitlab # [ 8.852191] systemd[1]: Starting linger-users.service... gitlab # [ 8.853086] systemd[1]: Stopping Name Service Cache Daemon (nsncd)... gitlab # [ 8.853996] systemd[1]: nscd.service: Deactivated successfully. gitlab # [ 8.854809] systemd[1]: Stopped Name Service Cache Daemon (nsncd). gitlab # [ 8.855700] systemd[1]: nscd.service: Consumed 20ms CPU time over 156ms wall clock time, 7.6M memory peak, 5.8M read from disk. gitlab # [ 8.863784] systemd-vconsole-setup[618]: Configuration of first virtual console was skipped, ignoring remaining ones. gitlab # [ 8.869508] systemd[1]: Starting Name Service Cache Daemon (nsncd)... gitlab # [ 8.872245] systemd[1]: Finished Virtual Console Setup. gitlab # [ 8.880051] systemd[1]: linger-users.service: Deactivated successfully. gitlab # [ 8.882074] systemd[1]: Finished linger-users.service. gitlab # [ 8.890094] nsncd[673]: Aug 15 13:20:36.720 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket" gitlab # [ 8.892579] systemd[1]: Finished resolvconf update. gitlab # [ 8.893439] systemd[1]: Started Name Service Cache Daemon (nsncd). gitlab # [ 8.894282] systemd[1]: Reached target Preparation for Network. gitlab # [ 8.895186] systemd[1]: Reached target Host and Network Name Lookups. gitlab # [ 8.896173] systemd[1]: Reached target User and Group Name Lookups. gitlab # [ 8.897810] systemd[1]: Starting DHCP Client... gitlab # [ 8.901097] systemd[1]: Starting Address configuration of eth1... gitlab # [ 8.903723] systemd[1]: Starting Extra networking commands.... gitlab # [ 8.934464] network-addresses-eth1-start[695]: adding address 192.168.1.1/24... done gitlab # [ 8.935743] network-addresses-eth1-start[695]: adding address 2001:db8:1::1/64... done gitlab # [ 8.942051] systemd[1]: Finished Address configuration of eth1. gitlab # [ 8.989529] systemd[1]: Finished Extra networking commands.. gitlab # [ 8.990747] systemd[1]: Reached target Network. gitlab # [ 8.994279] systemd[1]: Starting gitlab-workhorse.service... gitlab # [ 8.997116] systemd[1]: Starting Nginx Web Server... gitlab # [ 9.000675] dhcpcd[708]: dhcpcd-10.3.2 starting gitlab # [ 9.005547] dhcpcd[762]: dev: loaded udev gitlab # [ 9.006327] systemd[1]: Starting PostgreSQL Server... gitlab # [ 9.013335] systemd[1]: Starting Redis Server - redis-gitlab... gitlab # [ 9.020695] 8021q: 802.1Q VLAN Support v1.8 gitlab # [ 9.021152] 8021q: adding VLAN 0 to HW filter on device eth1 gitlab # [ 9.018982] systemd[1]: Starting Permit User Sessions... gitlab # [ 9.033679] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. gitlab # [ 9.057555] systemd[1]: Finished Permit User Sessions. gitlab # [ 9.061648] systemd[1]: Started Getty on tty1. gitlab # [ 9.063719] systemd[1]: Reached target Login Prompts. gitlab # [ 9.079762] cfg80211: Loading compiled-in X.509 certificates for regulatory database gitlab # [ 9.096881] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' gitlab # [ 9.097385] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' gitlab # [ 9.101610] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2 gitlab # [ 9.101950] cfg80211: failed to load regulatory.db gitlab # [ 9.143777] dhcpcd[762]: eth0: waiting for carrier[ 9.145457] 8021q: adding VLAN 0 to HW filter on device eth0 gitlab # gitlab # [ 9.146153] dhcpcd[762]: eth0: carrier acquired gitlab # [ 9.149379] dhcpcd[762]: DUID 00:01:00:01:32:13:24:24:52:54:00:12:34:56 gitlab # [ 9.150553] dhcpcd[762]: eth0: IAID 00:12:34:56 gitlab # [ 9.151235] dhcpcd[762]: eth0: adding address fe80::5054:ff:fe12:3456 gitlab # [ 9.193575] nginx-pre-start[788]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok gitlab # [ 9.195602] nginx-pre-start[788]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful gitlab # [ 9.197965] redis-server[785]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit. gitlab # [ 9.200367] redis-server[785]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo gitlab # [ 9.201840] redis-server[785]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=785, just started gitlab # [ 9.204272] redis-server[785]: Configuration loaded gitlab # [ 9.205300] redis[785]: monotonic clock: ARM CNTVCT @ 1000 ticks/us gitlab # [ 9.206342] systemd[1]: Started Nginx Web Server. gitlab # [ 9.207098] redis[785]: Running mode=standalone, port=0. gitlab # [ 9.208052] systemd[1]: Started Redis Server - redis-gitlab. gitlab # [ 9.209091] redis[785]: Server initialized gitlab # [ 9.209974] redis[785]: Ready to accept connections unix gitlab # [ 9.211052] redis[785]: WARNING: Redis does not require authentication. Redis will accept connections from any local client. gitlab # [ 9.226281] postgresql-pre-start[792]: The files belonging to this database system will be owned by user "postgres". gitlab # [ 9.228622] postgresql-pre-start[792]: This user must also own the server process. gitlab # [ 9.229841] postgresql-pre-start[792]: The database cluster will be initialized with locale "en_US.UTF-8". gitlab # [ 9.231241] postgresql-pre-start[792]: The default database encoding has accordingly been set to "UTF8". gitlab # [ 9.233563] postgresql-pre-start[792]: The default text search configuration will be set to "english". gitlab # [ 9.236263] postgresql-pre-start[792]: Data page checksums are enabled. gitlab # [ 9.237291] postgresql-pre-start[792]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok gitlab # [ 9.238611] postgresql-pre-start[792]: creating subdirectories ... ok gitlab # [ 9.239470] postgresql-pre-start[792]: selecting dynamic shared memory implementation ... posix gitlab # [ 9.270515] postgresql-pre-start[792]: selecting default "max_connections" ... 100 gitlab # [ 9.319819] postgresql-pre-start[792]: selecting default "shared_buffers" ... 128MB gitlab # [ 9.658214] systemd[1]: Started gitlab-workhorse.service. gitlab # [ 9.777163] postgresql-pre-start[792]: selecting default time zone ... UTC gitlab # [ 9.780944] postgresql-pre-start[792]: creating configuration files ... ok gitlab # [ 10.021028] postgresql-pre-start[792]: running bootstrap script ... ok gitlab # [ 10.025611] gitlab-workhorse[820]: time="2026-08-15T13:20:37Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4 gitlab # [ 10.027319] gitlab-workhorse[820]: time="2026-08-15T13:20:37Z" level=info msg="Using redis/go-redis" gitlab # [ 10.028824] gitlab-workhorse[820]: time="2026-08-15T13:20:37Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix gitlab # [ 10.169210] dhcpcd[762]: eth0: soliciting a DHCP lease gitlab # [ 10.181287] dhcpcd[762]: eth0: offered 10.0.2.15 from 10.0.2.2 gitlab # [ 10.200562] dhcpcd[762]: eth0: probing address 10.0.2.15/24 gitlab # [ 10.774341] postgresql-pre-start[792]: performing post-bootstrap initialization ... ok gitlab # [ 11.052652] postgresql-pre-start[792]: syncing data to disk ... ok gitlab # [ 11.055596] postgresql-pre-start[792]: initdb: warning: enabling "trust" authentication for local connections gitlab # [ 11.060069] postgresql-pre-start[792]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb. gitlab # [ 11.068074] postgresql-pre-start[792]: Success. You can now start the database server using: gitlab # [ 11.071814] postgresql-pre-start[792]: pg_ctl -D /var/lib/postgresql/18 -l logfile start gitlab # [ 11.281232] postgres[854]: [854] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit gitlab # [ 11.287083] postgres[854]: [854] LOG: listening on IPv6 address "::1", port 5432 gitlab # [ 11.291746] postgres[854]: [854] LOG: listening on IPv4 address "127.0.0.1", port 5432 gitlab # [ 11.296300] postgres[854]: [854] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432" gitlab # [ 11.300832] postgres[864]: [864] LOG: database system was shut down at 2026-08-15 13:20:38 GMT gitlab # [ 11.307975] postgres[854]: [854] LOG: database system is ready to accept connections gitlab # [ 11.316730] systemd[1]: Started PostgreSQL Server. gitlab # [ 11.319037] systemd[1]: Starting PostgreSQL Setup Scripts... gitlab # [ 11.428364] dhcpcd[762]: eth0: soliciting an IPv6 router gitlab # [ 11.430912] dhcpcd[762]: eth0: Router Advertisement from fe80::2 gitlab # [ 11.433730] dhcpcd[762]: eth0: adding address fec0::5054:ff:fe12:3456/64 gitlab # [ 11.436601] dhcpcd[762]: eth0: adding route to fec0::/64 gitlab # [ 11.438848] dhcpcd[762]: eth0: adding default route via fe80::2 gitlab # [ 11.731416] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[880]: mkdir -p /var/gitlab/state/home/.ssh: OK gitlab # [ 11.735954] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[880]: chmod 700 /var/gitlab/state/home/.ssh: OK gitlab # [ 11.738375] postgresql-setup-start[875]: CREATE DATABASE gitlab # [ 11.786646] postgresql-setup-start[891]: CREATE ROLE gitlab # [ 11.806580] postgresql-setup-start[896]: ALTER DATABASE gitlab # [ 11.826566] systemd[1]: Finished gitlab-config.service. gitlab # [ 11.836218] systemd[1]: Started gitaly.service. gitlab # [ 11.862545] postgresql-setup-start[905]: CREATE ROLE gitlab # [ 11.865787] systemd[1]: Finished PostgreSQL Setup Scripts. gitlab # [ 11.868217] systemd[1]: Reached target PostgreSQL. gitlab # [ 11.873295] systemd[1]: Starting gitlab-db-config.service... gitlab # [ 11.880226] systemd[1]: Starting gitlab-postgresql.service... gitlab # [ 12.097554] gitlab-postgresql-start[916]: CREATE DATABASE gitlab # [ 12.172912] gitlab-postgresql-start[930]: CREATE EXTENSION gitlab # [ 12.324162] gitlab-postgresql-start[937]: CREATE EXTENSION gitlab # [ 12.331960] systemd[1]: Finished gitlab-postgresql.service. gitlab # [ 12.676433] gitaly[904]: time="2026-08-15T13:20:40.505Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=904 gitlab # [ 12.692537] gitaly[904]: time="2026-08-15T13:20:40.506Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=904 gitlab # [ 12.700322] gitaly[904]: time="2026-08-15T13:20:40.506Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=904 gitlab # [ 12.707572] gitaly[904]: time="2026-08-15T13:20:40.507Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=904 gitlab # [ 12.716323] gitaly[904]: time="2026-08-15T13:20:40.507Z" level=info msg="Starting Gitaly" pid=904 version= gitlab # [ 12.958874] gitaly[904]: time="2026-08-15T13:20:40.789Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=281 pid=904 gitlab # [ 12.961355] gitaly[904]: time="2026-08-15T13:20:40.789Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=904 gitlab # [ 12.963808] gitaly[904]: time="2026-08-15T13:20:40.790Z" level=info msg="finished initializing command factory" duration_ms=0 pid=904 gitlab # [ 12.966339] gitaly[904]: time="2026-08-15T13:20:40.790Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-904/git-exec-947329305.d/git pid=904 gitlab # [ 12.969145] gitaly[904]: time="2026-08-15T13:20:40.794Z" level=info msg="finished detecting git version" duration_ms=4 pid=904 gitlab # [ 12.971484] gitaly[904]: time="2026-08-15T13:20:40.795Z" level=info msg="using Git version" pid=904 version=2.53.g gitlab # [ 12.973603] gitaly[904]: time="2026-08-15T13:20:40.795Z" level=info msg="clearing disk cache object folder" pid=904 storage=default gitlab # [ 12.975914] gitaly[904]: time="2026-08-15T13:20:40.795Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2937680691 pid=904 storage=default gitlab # [ 12.979832] gitaly[904]: time="2026-08-15T13:20:40.795Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2937680691 pid=904 storage=default gitlab # [ 12.984517] gitaly[904]: time="2026-08-15T13:20:40.796Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=904 gitlab # [ 12.989003] gitaly[904]: time="2026-08-15T13:20:40.796Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=904 gitlab # [ 12.993326] gitaly[904]: time="2026-08-15T13:20:40.796Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2937680691 clear_duration_ms=0 pid=904 storage=default gitlab # [ 12.997945] gitaly[904]: time="2026-08-15T13:20:40.817Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=904 gitlab # [ 13.001403] gitaly[904]: time="2026-08-15T13:20:40.817Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=904 schema=unix gitlab # [ 13.004646] gitaly[904]: time="2026-08-15T13:20:40.817Z" level=info msg="listening at address" address=/run/gitaly/gitaly-904/sock.d/intern pid=904 schema=unix gitlab # [ 13.007476] gitaly[904]: time="2026-08-15T13:20:40.817Z" level=info msg="finished tempdir cleaner walk" pid=904 storage=default time_ms=0 gitlab # [ 13.010187] gitaly[904]: time="2026-08-15T13:20:40.817Z" level=info msg="counting repositories" pid=904 gitlab # [ 13.012362] gitaly[904]: time="2026-08-15T13:20:40.818Z" level=info msg="starting to count repositories" pid=904 storage_path=/var/gitlab/state/repositories gitlab # [ 13.015112] gitaly[904]: time="2026-08-15T13:20:40.818Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=904 storage_path=/var/gitlab/state/repositories gitlab # [ 13.018546] gitaly[904]: time="2026-08-15T13:20:40.818Z" level=info msg="completed counting all repositories" counting_duration=0 pid=904 gitlab # [ 13.021361] gitaly[904]: time="2026-08-15T13:20:40.820Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=904 gitlab # [ 13.024108] gitaly[904]: time="2026-08-15T13:20:40.820Z" level=info msg="Started Gitaly" duration_ms=312 pid=904 gitlab # [ 13.026488] gitaly[904]: time="2026-08-15T13:20:40.820Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=904 gitlab # [ 13.029133] gitaly[904]: time="2026-08-15T13:20:40.820Z" level=info msg="maintenance: daily scheduled" pid=904 scheduled="2026-08-16 12:00:00 +0000 UTC" gitlab # [ 13.637270] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.650585] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 13.669713] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 15.036813] dhcpcd[762]: eth0: leased 10.0.2.15 for 86400 seconds gitlab # [ 15.037157] dhcpcd[762]: eth0: adding route to 10.0.2.0/24 gitlab # [ 15.037394] dhcpcd[762]: eth0: adding default route via 10.0.2.2 gitlab # [ 15.254651] systemd[1]: Started DHCP Client. gitlab # [ 15.258047] systemd[1]: Reached target Network is Online. gitlab # [ 16.317292] gitaly[904]: time="2026-08-15T13:20:44.146Z" level=info msg="License database preloaded" duration_ms=3639 pid=904 gitlab # [ 60.114122] gitlab-workhorse[820]: time="2026-08-15T13:21:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 60.232108] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[911]: W, [2026-08-15T13:21:28.061622 #911] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 63.246806] postgres[1458]: [1458] ERROR: relation "feature_gates" does not exist at character 523 gitlab # [ 63.247199] postgres[1458]: [1458] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod), gitlab # [ 63.247322] postgres[1458]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod, gitlab # [ 63.247463] postgres[1458]: c.collname, col_description(a.attrelid, a.attnum) AS comment, gitlab # [ 63.247580] postgres[1458]: attidentity AS identity, gitlab # [ 63.247696] postgres[1458]: attgenerated as attgenerated gitlab # [ 63.247815] postgres[1458]: FROM pg_attribute a gitlab # [ 63.247929] postgres[1458]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum gitlab # [ 63.254619] postgres[1458]: LEFT JOIN pg_type t ON a.atttypid = t.oid gitlab # [ 63.272750] postgres[1458]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation gitlab # [ 63.274114] postgres[1458]: WHERE a.attrelid = '"feature_gates"'::regclass gitlab # [ 63.275039] postgres[1458]: AND a.attnum > 0 AND NOT a.attisdropped gitlab # [ 63.275908] postgres[1458]: ORDER BY a.attnum gitlab # [ 63.276603] postgres[1458]: gitlab # [ 68.506887] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1504]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping gitlab # [ 68.507312] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1504]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping gitlab # [ 70.122664] gitlab-workhorse[820]: time="2026-08-15T13:21:37Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 80.130306] gitlab-workhorse[820]: time="2026-08-15T13:21:47Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 90.141932] gitlab-workhorse[820]: time="2026-08-15T13:21:57Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 95.446440] postgres[1612]: [1612] ERROR: relation "table_without_models" does not exist gitlab # [ 95.446988] postgres[1612]: [1612] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 100.152763] gitlab-workhorse[820]: time="2026-08-15T13:22:07Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 110.162379] gitlab-workhorse[820]: time="2026-08-15T13:22:17Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 120.170426] gitlab-workhorse[820]: time="2026-08-15T13:22:27Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 126.634889] systemd[1]: Finished gitlab-db-config.service. gitlab # [ 126.641461] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 126.646894] systemd[1]: Starting gitlab.service... gitlab # [ 127.198908] bundle[1996]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 127.205339] bundle[1996]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 127.211208] bundle[1996]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 130.173306] gitlab-workhorse[820]: time="2026-08-15T13:22:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 132.082529] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.881Z","pid":1996,"message":"* Enabling systemd notification integration"} gitlab # [ 132.084766] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"Puma starting in cluster mode..."} gitlab # [ 132.086282] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"} gitlab # [ 132.087743] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 132.089742] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Min threads: 0"} gitlab # [ 132.090962] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Max threads: 4"} gitlab # [ 132.092184] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Environment: production"} gitlab # [ 132.093480] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Master PID: 1996"} gitlab # [ 132.094715] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.915Z","pid":1996,"message":"* Workers: 2"} gitlab # [ 132.095932] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.916Z","pid":1996,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"} gitlab # [ 132.097692] bundle[1996]: {"timestamp":"2026-08-15T13:22:39.916Z","pid":1996,"message":"* Preloading application"} gitlab # [ 132.243697] sidekiq-cluster[1995]: {"severity":"INFO","time":"2026-08-15T13:22:40.031Z","message":"Starting cluster with 1 processes"} gitlab # [ 140.174750] gitlab-workhorse[820]: time="2026-08-15T13:22:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 150.182422] gitlab-workhorse[820]: time="2026-08-15T13:22:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 160.190344] gitlab-workhorse[820]: time="2026-08-15T13:23:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 170.200513] gitlab-workhorse[820]: time="2026-08-15T13:23:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 171.560787] bundle[1996]: W, [2026-08-15T13:23:19.390208 #1996] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 175.624950] sidekiq-cluster[2046]: 2026-08-15T13:23:23.454Z pid=2046 tid=wnq INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>2046, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>426147840} gitlab # [ 175.921603] sidekiq-cluster[2046]: W, [2026-08-15T13:23:23.752011 #2046] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 176.073314] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:23.903Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab # [ 179.986434] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.815Z","message":"GitLab reliable fetch activated!"} gitlab # [ 179.986796] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.816Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 180.020416] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.851Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"} gitlab # [ 180.027391] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.854Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"} gitlab # [ 180.034558] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.857Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"} gitlab # [ 180.042394] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.860Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"} gitlab # [ 180.049858] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.863Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"} gitlab # [ 180.056717] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.865Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"} gitlab # [ 180.063511] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.870Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"} gitlab # [ 180.069278] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.875Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"} gitlab # [ 180.074221] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.879Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"} gitlab # [ 180.078858] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.884Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"} gitlab # [ 180.083767] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.888Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"} gitlab # [ 180.088161] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.895Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"} gitlab # [ 180.091759] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.899Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"} gitlab # [ 180.095731] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.903Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"} gitlab # [ 180.099420] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.909Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"} gitlab # [ 180.102922] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.912Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"} gitlab # [ 180.106067] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.919Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"} gitlab # [ 180.108964] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.924Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"} gitlab # [ 180.112053] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.929Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"} gitlab # [ 180.114847] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.935Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"} gitlab # [ 180.118035] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.941Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"} gitlab # [ 180.121203] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.945Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"} gitlab # [ 180.123799] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.949Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"} gitlab # [ 180.126617] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.954Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"} gitlab # [ 180.129323] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.958Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"} gitlab # [ 180.132594] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.963Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"} gitlab # [ 180.137069] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.967Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"} gitlab # [ 180.141401] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.972Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"} gitlab # [ 180.147490] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.977Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"} gitlab # [ 180.156489] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.986Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"} gitlab # [ 180.162949] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:27.993Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"} gitlab # [ 180.177427] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.003Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"} gitlab # [ 180.181572] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.010Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"} gitlab # [ 180.185561] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.014Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"} gitlab # [ 180.189199] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.017Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"} gitlab # [ 180.192689] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.020Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"} gitlab # [ 180.196502] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.024Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"} gitlab # [ 180.199484] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.027Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"} gitlab # [ 180.203082] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.030Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"} gitlab # [ 180.206260] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.033Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"} gitlab # [ 180.209625] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.036Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"} gitlab # [ 180.212746] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.038Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"} gitlab # [ 180.215776] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.041Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"} gitlab # [ 180.218769] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.043Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"} gitlab # [ 180.222273] gitlab-workhorse[820]: time="2026-08-15T13:23:28Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 180.226110] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.046Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"} gitlab # [ 180.228978] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.049Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"} gitlab # [ 180.231536] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.052Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"} gitlab # [ 180.234165] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.054Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"} gitlab # [ 180.236652] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.057Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"} gitlab # [ 180.239001] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.059Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"} gitlab # [ 180.242522] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.062Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"} gitlab # [ 180.244822] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.065Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"} gitlab # [ 180.247106] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.074Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"} gitlab # [ 180.249792] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.077Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"} gitlab # [ 180.252087] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.080Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"} gitlab # [ 180.254330] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.082Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"} gitlab # [ 180.256696] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.085Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"} gitlab # [ 180.259082] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.088Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"} gitlab # [ 180.261581] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.090Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"} gitlab # [ 180.264053] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.092Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"} gitlab # [ 180.266235] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.095Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"} gitlab # [ 180.268451] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.097Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"} gitlab # [ 180.346284] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.176Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"} gitlab # [ 180.350416] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.180Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"} gitlab # [ 180.354559] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.183Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"} gitlab # [ 180.358622] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.188Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"} gitlab # [ 180.362472] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.190Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"} gitlab # [ 180.366562] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.194Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"} gitlab # [ 180.370827] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.196Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"} gitlab # [ 180.375163] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.199Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"} gitlab # [ 180.379348] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.201Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"} gitlab # [ 180.383498] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.203Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"} gitlab # [ 180.387542] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.206Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"} gitlab # [ 180.391566] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.208Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"} gitlab # [ 180.395367] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.211Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"} gitlab # [ 180.399030] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.213Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"} gitlab # [ 180.402328] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.216Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"} gitlab # [ 180.405844] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.218Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"} gitlab # [ 180.408798] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.221Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"} gitlab # [ 180.411532] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.223Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"} gitlab # [ 180.414965] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.225Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"} gitlab # [ 180.418112] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.228Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"} gitlab # [ 180.420927] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.231Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"} gitlab # [ 180.423452] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.234Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"} gitlab # [ 180.426287] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.236Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"} gitlab # [ 180.428801] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.239Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"} gitlab # [ 180.431292] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.242Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"} gitlab # [ 180.433872] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.245Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"} gitlab # [ 180.436393] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.247Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"} gitlab # [ 180.438781] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.250Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"} gitlab # [ 180.441264] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.251Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"} gitlab # [ 180.443530] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.254Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"} gitlab # [ 180.447064] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.256Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"} gitlab # [ 180.449619] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.258Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"} gitlab # [ 180.452781] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.261Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"} gitlab # [ 180.455309] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.265Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"} gitlab # [ 180.457862] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.267Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"} gitlab # [ 180.460248] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.270Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"} gitlab # [ 180.462512] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.284Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"} gitlab # [ 180.464738] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.287Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"} gitlab # [ 180.466821] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.289Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"} gitlab # [ 180.469088] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.293Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"} gitlab # [ 180.471041] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.296Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"} gitlab # [ 180.473302] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.299Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"} gitlab # [ 180.475831] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.302Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"} gitlab # [ 180.478500] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.305Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"} gitlab # [ 180.480649] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.307Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"} gitlab # [ 180.482894] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.310Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"} gitlab # [ 180.485200] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:23:28.313Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"} gitlab # [ 190.212855] gitlab-workhorse[820]: time="2026-08-15T13:23:38Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 200.213755] gitlab-workhorse[820]: time="2026-08-15T13:23:48Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 210.215169] gitlab-workhorse[820]: time="2026-08-15T13:23:58Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 219.809912] postgres[2783]: [2783] ERROR: relation "table_without_models" does not exist gitlab # [ 219.810321] postgres[2783]: [2783] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 220.222507] gitlab-workhorse[820]: time="2026-08-15T13:24:08Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 224.281284] postgres[2821]: [2821] ERROR: relation "table_without_models" does not exist gitlab # [ 224.284852] postgres[2821]: [2821] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 225.010455] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:12.839Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 225.025065] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:12.855Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 225.031444] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:12.855Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 225.037009] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:12.855Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 225.043135] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:12.857Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab # [ 227.842386] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:15.670Z","message":"Cleaning working queues"} gitlab # [ 227.851302] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:15.681Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"cb009a15b1ad540af12e38aa","created_at":"2026-08-15T13:22:33.952Z","trace_propagation_headers":{"sentry-trace":"2ec887ba6e074d448b041c5fad45c85e-171b088b7823428d","baggage":"sentry-trace_id=2ec887ba6e074d448b041c5fad45c85e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ac36ea6a195d26aa87021efa1c867d7c","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-15T13:22:33.965Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-cb009a15b1ad540af12e38aa: start","job_status":"start","queue_duration_s":101.715524,"scheduling_latency_s":101.715634} gitlab # [ 228.314909] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:16.144Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"cb009a15b1ad540af12e38aa","created_at":"2026-08-15T13:22:33.952Z","trace_propagation_headers":{"sentry-trace":"2ec887ba6e074d448b041c5fad45c85e-171b088b7823428d","baggage":"sentry-trace_id=2ec887ba6e074d448b041c5fad45c85e,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"ac36ea6a195d26aa87021efa1c867d7c","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-15T13:22:33.965Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-cb009a15b1ad540af12e38aa: done: 0.463952 sec","job_status":"done","queue_duration_s":101.715524,"scheduling_latency_s":101.715634,"redis_calls":29,"redis_duration_s":0.015941,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.007835,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00331,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004796,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.023,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.222792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.23453236,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.463952,"completed_at":"2026-08-15T13:24:16.144Z","load_balancing_strategy":"primary","db_duration_s":0.118043,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 230.230177] gitlab-workhorse[820]: time="2026-08-15T13:24:18Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri= gitlab # [ 231.420826] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.250Z","pid":1996,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"} gitlab # [ 231.421342] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.250Z","pid":1996,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"} gitlab # [ 231.421527] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.250Z","pid":1996,"message":"! #\u003cThread:0x0000e520f8f05c90 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"} gitlab # [ 231.421658] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.250Z","pid":1996,"message":"! #\u003cThread:0x0000e520f888a028 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"} gitlab # [ 231.421777] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.250Z","pid":1996,"message":"Use Ctrl-C to stop"} gitlab # [ 231.421892] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.251Z","pid":1996,"message":"Sending status to systemd every 1.0 sec"} gitlab # [ 232.134396] bundle[1996]: {"timestamp":"2026-08-15T13:24:19.964Z","pid":1996,"message":"- Worker 0 (PID: 2948) booted in 0.67s, phase: 0"} gitlab # [ 232.175489] bundle[1996]: {"timestamp":"2026-08-15T13:24:20.005Z","pid":1996,"message":"- Worker 1 (PID: 2952) booted in 0.69s, phase: 0"} gitlab # [ 232.182030] systemd[1]: Started gitlab.service. gitlab # [ 232.184313] systemd[1]: Reached target Common target for all GitLab services.. gitlab # [ 232.189877] systemd[1]: Starting setup-gitlab.service... gitlab # [ 232.231947] setup-gitlab-start[2976]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done' gitlab # [ 235.876357] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:24:23 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 3595 gitlab # [ 235.882138] setup-gitlab-start[2976]: + TOKEN=nixbot-test-token-of-20-chars gitlab # [ 235.883210] setup-gitlab-start[2976]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n' gitlab # [ 235.903052] runuser[3002]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0) gitlab # [ 236.433840] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 236.440542] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 236.447034] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 237.299506] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 237.299883] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 237.302480] setup-gitlab-start[3003]: Source locally installed gems is ignoring # because it is missing extensions gitlab # [ 238.867061] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:26.695Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a11925f1f1541b24fd2db013","created_at":"2026-08-15T13:24:26.600Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b8c922627a1d7176eb2057e756d7f91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.692Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a11925f1f1541b24fd2db013: start","job_status":"start","queue_duration_s":0.003665,"scheduling_latency_s":0.003726} gitlab # [ 239.368315] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.197Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"a11925f1f1541b24fd2db013","created_at":"2026-08-15T13:24:26.600Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7b8c922627a1d7176eb2057e756d7f91","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.692Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-a11925f1f1541b24fd2db013: done: 0.501589 sec","job_status":"done","queue_duration_s":0.003665,"scheduling_latency_s":0.003726,"redis_calls":14,"redis_duration_s":0.033197,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.0071,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.016331,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009766,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.086,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.041,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0029999,"exclusive_lock_hold_duration_s":0.26023048,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.501589,"completed_at":"2026-08-15T13:24:27.197Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.107053,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.473950] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.212Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"f4c970bbfcae35d42e28dfb2","created_at":"2026-08-15T13:24:26.709Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c268783755bf1f8b0501c4bce24a78","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.723Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-f4c970bbfcae35d42e28dfb2: start","job_status":"start","queue_duration_s":0.488501,"scheduling_latency_s":0.488569} gitlab # [ 239.495053] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.318Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"f4c970bbfcae35d42e28dfb2","created_at":"2026-08-15T13:24:26.709Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41c268783755bf1f8b0501c4bce24a78","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.723Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-f4c970bbfcae35d42e28dfb2: done: 0.106715 sec","job_status":"done","queue_duration_s":0.488501,"scheduling_latency_s":0.488569,"redis_calls":12,"redis_duration_s":0.030875,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.001337,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004122,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009178,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.016238,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.042887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00079102,"duration_s":0.106715,"completed_at":"2026-08-15T13:24:27.318Z","load_balancing_strategy":"primary","db_duration_s":0.003742,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.542974] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.345Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b8d9ff13c8a118e868809a8","created_at":"2026-08-15T13:24:26.732Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd0839b74311b7f8ab4f9c6f5cdb3946","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.745Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b8d9ff13c8a118e868809a8: start","job_status":"start","queue_duration_s":0.600287,"scheduling_latency_s":0.600352} gitlab # [ 239.616618] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.446Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 239.684178] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.513Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8b8d9ff13c8a118e868809a8","created_at":"2026-08-15T13:24:26.732Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"cd0839b74311b7f8ab4f9c6f5cdb3946","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.745Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8b8d9ff13c8a118e868809a8: done: 0.167682 sec","job_status":"done","queue_duration_s":0.600287,"scheduling_latency_s":0.600352,"redis_calls":10,"redis_duration_s":0.044397,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006193,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.010786,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.027418,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00231552,"duration_s":0.167682,"completed_at":"2026-08-15T13:24:27.513Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002346,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.777641] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.521Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"35aaa28a1ad84109a01cee83","created_at":"2026-08-15T13:24:26.768Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c955135e26f8fd9f30c9727565eeaa59","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.789Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-35aaa28a1ad84109a01cee83: start","job_status":"start","queue_duration_s":0.732396,"scheduling_latency_s":0.732461} gitlab # [ 239.883224] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.711Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"35aaa28a1ad84109a01cee83","created_at":"2026-08-15T13:24:26.768Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c955135e26f8fd9f30c9727565eeaa59","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.789Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-35aaa28a1ad84109a01cee83: done: 0.189724 sec","job_status":"done","queue_duration_s":0.732396,"scheduling_latency_s":0.732461,"redis_calls":10,"redis_duration_s":0.020799,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00855,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.009386,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002863,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.04851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00371602,"duration_s":0.189724,"completed_at":"2026-08-15T13:24:27.710Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049098,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 239.968009] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.727Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"87933c4bac72fff996f7286f","created_at":"2026-08-15T13:24:26.813Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f40898be83cb2791fd936c97f0f623d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.824Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-87933c4bac72fff996f7286f: start","job_status":"start","queue_duration_s":0.902897,"scheduling_latency_s":0.902952} gitlab # [ 240.058969] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.888Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"87933c4bac72fff996f7286f","created_at":"2026-08-15T13:24:26.813Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f40898be83cb2791fd936c97f0f623d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.824Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-87933c4bac72fff996f7286f: done: 0.161005 sec","job_status":"done","queue_duration_s":0.902897,"scheduling_latency_s":0.902952,"redis_calls":12,"redis_duration_s":0.052128999999999995,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.004163,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.041772,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.006194,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.051066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0020909,"exclusive_lock_hold_duration_s":0.030918420000000002,"duration_s":0.161005,"completed_at":"2026-08-15T13:24:27.888Z","load_balancing_strategy":"primary","db_duration_s":0.021909,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.134284] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:27.901Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"215be01857f2289e6104e603","created_at":"2026-08-15T13:24:26.874Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e38a8635d306aaaa87f177835de0425","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.901Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-215be01857f2289e6104e603: start","job_status":"start","queue_duration_s":0.999703,"scheduling_latency_s":0.999755} gitlab # [ 240.227590] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"215be01857f2289e6104e603","created_at":"2026-08-15T13:24:26.874Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e38a8635d306aaaa87f177835de0425","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.901Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-215be01857f2289e6104e603: done: 0.155746 sec","job_status":"done","queue_duration_s":0.999703,"scheduling_latency_s":0.999755,"redis_calls":10,"redis_duration_s":0.032704,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.005422,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.022591,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004691,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.041,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.041463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00210866,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.155746,"completed_at":"2026-08-15T13:24:28.056Z","load_balancing_strategy":"primary","db_duration_s":0.053393,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.320245] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.072Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"32165c587484379314a85cd9","created_at":"2026-08-15T13:24:26.945Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0c9fc8fedc6c140aff6e8827350260","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.958Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-32165c587484379314a85cd9: start","job_status":"start","queue_duration_s":1.114173,"scheduling_latency_s":1.11423} gitlab # [ 240.548630] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"32165c587484379314a85cd9","created_at":"2026-08-15T13:24:26.945Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bf0c9fc8fedc6c140aff6e8827350260","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.958Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-32165c587484379314a85cd9: done: 0.306408 sec","job_status":"done","queue_duration_s":1.114173,"scheduling_latency_s":1.11423,"redis_calls":14,"redis_duration_s":0.004333,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001398,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001542,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001393,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.058,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.058,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.228432,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00118548,"exclusive_lock_hold_duration_s":0.2404636,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.306408,"completed_at":"2026-08-15T13:24:28.378Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.072999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.647250] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.380Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a95ddf2a5ea0fe2f768d046c","created_at":"2026-08-15T13:24:26.966Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd3b8417bf469dd1f1ba642d20ad656b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.980Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a95ddf2a5ea0fe2f768d046c: start","job_status":"start","queue_duration_s":1.400094,"scheduling_latency_s":1.400119} gitlab # [ 240.666699] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.394Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 240.671311] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.403Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"a95ddf2a5ea0fe2f768d046c","created_at":"2026-08-15T13:24:26.966Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd3b8417bf469dd1f1ba642d20ad656b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:26.980Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-a95ddf2a5ea0fe2f768d046c: done: 0.02314 sec","job_status":"done","queue_duration_s":1.400094,"scheduling_latency_s":1.400119,"redis_calls":10,"redis_duration_s":0.002601,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000575,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001502,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000524,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012799,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039854,"duration_s":0.02314,"completed_at":"2026-08-15T13:24:28.403Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 240.721278] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"00cb163e69bfdcf9d213a234","created_at":"2026-08-15T13:24:27.016Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98da4a6711e34b6460368cb0e2d5a02d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.026Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-00cb163e69bfdcf9d213a234: start","job_status":"start","queue_duration_s":1.378701,"scheduling_latency_s":1.378726} gitlab # [ 241.072281] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.900Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"00cb163e69bfdcf9d213a234","created_at":"2026-08-15T13:24:27.016Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"98da4a6711e34b6460368cb0e2d5a02d","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.026Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-00cb163e69bfdcf9d213a234: done: 0.495468 sec","job_status":"done","queue_duration_s":1.378701,"scheduling_latency_s":1.378726,"redis_calls":565,"redis_duration_s":0.120214,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001036,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001894,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.117284,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.427306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00047276,"duration_s":0.495468,"completed_at":"2026-08-15T13:24:28.900Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000679,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.163024] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.903Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"22350fc9993d558d71b10e3f","created_at":"2026-08-15T13:24:27.037Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77adea9c4e6aee2da34e7ac430687d9b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.046Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-22350fc9993d558d71b10e3f: start","job_status":"start","queue_duration_s":1.856694,"scheduling_latency_s":1.856722} gitlab # [ 241.178413] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.935Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ObjectStorage::DeleteStaleDirectUploadsWorker","jid":"22350fc9993d558d71b10e3f","created_at":"2026-08-15T13:24:27.037Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"77adea9c4e6aee2da34e7ac430687d9b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:da92be7816ff691bea6b0627c50cb73b98e4b5009a988cab37d277c0df9aa326","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.046Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ObjectStorage::DeleteStaleDirectUploadsWorker JID-22350fc9993d558d71b10e3f: done: 0.031986 sec","job_status":"done","queue_duration_s":1.856694,"scheduling_latency_s":1.856722,"redis_calls":12,"redis_duration_s":0.0045060000000000005,"redis_read_bytes":11,"redis_write_bytes":1247,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000702,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":333,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001729,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002075,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":296,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017958,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00041672,"extra.object_storage_delete_stale_direct_uploads_worker.total_pending_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.total_deleted_stale_entries":0,"extra.object_storage_delete_stale_direct_uploads_worker.execution_timeout":false,"duration_s":0.031986,"completed_at":"2026-08-15T13:24:28.935Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001224,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.224123] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.937Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1ff12bd1581b0e738fd59242","created_at":"2026-08-15T13:24:27.052Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7808d7b03bad55ec1310ea146bc43386","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.068Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1ff12bd1581b0e738fd59242: start","job_status":"start","queue_duration_s":1.869705,"scheduling_latency_s":1.869731} gitlab # [ 241.238433] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"1ff12bd1581b0e738fd59242","created_at":"2026-08-15T13:24:27.052Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7808d7b03bad55ec1310ea146bc43386","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.068Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-1ff12bd1581b0e738fd59242: done: 0.032198 sec","job_status":"done","queue_duration_s":1.869705,"scheduling_latency_s":1.869731,"redis_calls":12,"redis_duration_s":0.003858,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00085,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001622,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001386,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0007773,"exclusive_lock_hold_duration_s":0.00516502,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.032198,"completed_at":"2026-08-15T13:24:28.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003261,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.283109] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:28.973Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e5a7ac19830e5c0eb3fbf92f","created_at":"2026-08-15T13:24:27.093Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f939b920ce8c952f1ee1ebe6221891e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.104Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e5a7ac19830e5c0eb3fbf92f: start","job_status":"start","queue_duration_s":1.868297,"scheduling_latency_s":1.868342} gitlab # [ 241.297956] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e5a7ac19830e5c0eb3fbf92f","created_at":"2026-08-15T13:24:27.093Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f939b920ce8c952f1ee1ebe6221891e2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.104Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e5a7ac19830e5c0eb3fbf92f: done: 0.059189 sec","job_status":"done","queue_duration_s":1.868297,"scheduling_latency_s":1.868342,"redis_calls":12,"redis_duration_s":0.0047279999999999996,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001633,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002281,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036811,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0007413,"duration_s":0.059189,"completed_at":"2026-08-15T13:24:29.032Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001999,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.341339] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.035Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"bd59d9f898a01a3e9f83c6df","created_at":"2026-08-15T13:24:27.123Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.149Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-bd59d9f898a01a3e9f83c6df: start","job_status":"start","queue_duration_s":1.886032,"scheduling_latency_s":1.886066} gitlab # [ 241.394515] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.224Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"bd59d9f898a01a3e9f83c6df","created_at":"2026-08-15T13:24:27.123Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.149Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-bd59d9f898a01a3e9f83c6df: done: 0.189272 sec","job_status":"done","queue_duration_s":1.886032,"scheduling_latency_s":1.886066,"redis_calls":71,"redis_duration_s":0.027801,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.004755,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000841,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.021188,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.159738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029174,"duration_s":0.189272,"completed_at":"2026-08-15T13:24:29.224Z","load_balancing_strategy":"primary","db_duration_s":0.004635,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.447254] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f96d17b549a3784a9feda430","created_at":"2026-08-15T13:24:27.172Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5983e0c53d06577b0eb99853922d3882","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:24:27.207Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f96d17b549a3784a9feda430: start","job_status":"start","queue_duration_s":2.019999,"scheduling_latency_s":2.020029} gitlab # [ 241.460767] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.270Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"f96d17b549a3784a9feda430","created_at":"2026-08-15T13:24:27.172Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5983e0c53d06577b0eb99853922d3882","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:24:27.207Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-f96d17b549a3784a9feda430: done: 0.043483 sec","job_status":"done","queue_duration_s":2.019999,"scheduling_latency_s":2.020029,"redis_calls":10,"redis_duration_s":0.003011,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000601,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00179,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00062,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026324,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00054724,"duration_s":0.043483,"completed_at":"2026-08-15T13:24:29.270Z","load_balancing_strategy":"primary","db_duration_s":0.007726,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.503404] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.273Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"66fae340253881a585559c3b","created_at":"2026-08-15T13:24:27.252Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62d52d178f755f437ca8f722686fa280","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.312Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-66fae340253881a585559c3b: start","job_status":"start","queue_duration_s":1.960209,"scheduling_latency_s":1.96024} gitlab # [ 241.523520] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"66fae340253881a585559c3b","created_at":"2026-08-15T13:24:27.252Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"62d52d178f755f437ca8f722686fa280","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.312Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-66fae340253881a585559c3b: done: 0.080641 sec","job_status":"done","queue_duration_s":1.960209,"scheduling_latency_s":1.96024,"redis_calls":12,"redis_duration_s":0.004791,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000772,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002073,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001946,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.032,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034002,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00112224,"exclusive_lock_hold_duration_s":0.0291369,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.080641,"completed_at":"2026-08-15T13:24:29.353Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.035929,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.568894] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b9b3ac6049b0777a8a5190cc","created_at":"2026-08-15T13:24:27.348Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41dcbc454e039bf05685dcb5f3291a4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.366Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b9b3ac6049b0777a8a5190cc: start","job_status":"start","queue_duration_s":1.990317,"scheduling_latency_s":1.990354} gitlab # [ 241.583340] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.378Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 241.586628] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"b9b3ac6049b0777a8a5190cc","created_at":"2026-08-15T13:24:27.348Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41dcbc454e039bf05685dcb5f3291a4f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.366Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-b9b3ac6049b0777a8a5190cc: done: 0.032731 sec","job_status":"done","queue_duration_s":1.990317,"scheduling_latency_s":1.990354,"redis_calls":10,"redis_duration_s":0.003725,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000756,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.0023,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000669,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01837,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00076262,"duration_s":0.032731,"completed_at":"2026-08-15T13:24:29.389Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001071,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.629245] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"492fe242a7069889b36f3010","created_at":"2026-08-15T13:24:27.416Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4287eebed9de3da0d1776396d8540f15","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.443Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-492fe242a7069889b36f3010: start","job_status":"start","queue_duration_s":1.947976,"scheduling_latency_s":1.948003} gitlab # [ 241.643841] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"492fe242a7069889b36f3010","created_at":"2026-08-15T13:24:27.416Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4287eebed9de3da0d1776396d8540f15","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.443Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-492fe242a7069889b36f3010: done: 0.042278 sec","job_status":"done","queue_duration_s":1.947976,"scheduling_latency_s":1.948003,"redis_calls":10,"redis_duration_s":0.004016,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000673,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002604,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000739,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024203,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00060076,"duration_s":0.042278,"completed_at":"2026-08-15T13:24:29.433Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006818,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.687929] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.436Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3fbe2ac9b070b120347745e4","created_at":"2026-08-15T13:24:27.504Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9e67761cd9c8cd7ce76a668f537cd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:24:27.530Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3fbe2ac9b070b120347745e4: start","job_status":"start","queue_duration_s":1.906047,"scheduling_latency_s":1.906075} gitlab # [ 241.701617] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.452Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 241.704897] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.461Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"3fbe2ac9b070b120347745e4","created_at":"2026-08-15T13:24:27.504Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0e9e67761cd9c8cd7ce76a668f537cd5","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:24:27.530Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-3fbe2ac9b070b120347745e4: done: 0.02518 sec","job_status":"done","queue_duration_s":1.906047,"scheduling_latency_s":1.906075,"redis_calls":10,"redis_duration_s":0.002958,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000659,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001706,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003975,"duration_s":0.02518,"completed_at":"2026-08-15T13:24:29.461Z","load_balancing_strategy":"primary","db_duration_s":0.000905,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.746133] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.463Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"17ddb581a04b2c3cf9b1fffa","created_at":"2026-08-15T13:24:27.577Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"926b1d6cb4f75f57e40036cb2f17e51d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.614Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-17ddb581a04b2c3cf9b1fffa: start","job_status":"start","queue_duration_s":1.848891,"scheduling_latency_s":1.848911} gitlab # [ 241.760716] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"17ddb581a04b2c3cf9b1fffa","created_at":"2026-08-15T13:24:27.577Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"926b1d6cb4f75f57e40036cb2f17e51d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.614Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-17ddb581a04b2c3cf9b1fffa: done: 0.043825 sec","job_status":"done","queue_duration_s":1.848891,"scheduling_latency_s":1.848911,"redis_calls":10,"redis_duration_s":0.0037020000000000004,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000863,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002272,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000567,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027808,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00062988,"duration_s":0.043825,"completed_at":"2026-08-15T13:24:29.507Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000831,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.803743] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fda53c0174bb825fe4e45aee","created_at":"2026-08-15T13:24:27.639Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2381726e789a30f25abeed0de0842e9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.658Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fda53c0174bb825fe4e45aee: start","job_status":"start","queue_duration_s":1.85222,"scheduling_latency_s":1.852264} gitlab # [ 241.818412] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.543Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 241.822071] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"fda53c0174bb825fe4e45aee","created_at":"2026-08-15T13:24:27.639Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2381726e789a30f25abeed0de0842e9f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.658Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-fda53c0174bb825fe4e45aee: done: 0.050788 sec","job_status":"done","queue_duration_s":1.85222,"scheduling_latency_s":1.852264,"redis_calls":10,"redis_duration_s":0.004502,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000824,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002953,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000725,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030909,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0006194,"duration_s":0.050788,"completed_at":"2026-08-15T13:24:29.561Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001083,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.864501] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.564Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ec2960caf698ea653fec6379","created_at":"2026-08-15T13:24:27.704Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65bbb0194fc61811fba59b9b55323921","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.731Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ec2960caf698ea653fec6379: start","job_status":"start","queue_duration_s":1.833089,"scheduling_latency_s":1.833128} gitlab # [ 241.898518] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.728Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ec2960caf698ea653fec6379","created_at":"2026-08-15T13:24:27.704Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"65bbb0194fc61811fba59b9b55323921","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.731Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ec2960caf698ea653fec6379: done: 0.164579 sec","job_status":"done","queue_duration_s":1.833089,"scheduling_latency_s":1.833128,"redis_calls":12,"redis_duration_s":0.003613,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000479,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001465,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001669,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.143767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00082674,"exclusive_lock_hold_duration_s":0.00896596,"duration_s":0.164579,"completed_at":"2026-08-15T13:24:29.728Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.941169] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.730Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"548e874236c3e5851bd85d94","created_at":"2026-08-15T13:24:27.786Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1064ba94f5254b90a94eeab6e170d15d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:24:27.818Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-548e874236c3e5851bd85d94: start","job_status":"start","queue_duration_s":1.911954,"scheduling_latency_s":1.911977} gitlab # [ 241.954545] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.743Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 241.958381] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.751Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"548e874236c3e5851bd85d94","created_at":"2026-08-15T13:24:27.786Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1064ba94f5254b90a94eeab6e170d15d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:24:27.818Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-548e874236c3e5851bd85d94: done: 0.02122 sec","job_status":"done","queue_duration_s":1.911954,"scheduling_latency_s":1.911977,"redis_calls":10,"redis_duration_s":0.001376,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000302,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000781,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000293,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011582,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027982,"duration_s":0.02122,"completed_at":"2026-08-15T13:24:29.751Z","load_balancing_strategy":"primary","db_duration_s":0.001654,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 241.999437] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"f0b8197d61b7177431f1b57f","created_at":"2026-08-15T13:24:27.839Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c24b80157b58b2eb0f673b7768f73da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.870Z","job_size_bytes":92,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-f0b8197d61b7177431f1b57f: start","job_status":"start","queue_duration_s":1.882578,"scheduling_latency_s":1.882598} gitlab # [ 242.014212] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.841Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["[FILTERED]"],"class":"Database::BackgroundOperation::CronEnqueueWorker","jid":"f0b8197d61b7177431f1b57f","created_at":"2026-08-15T13:24:27.839Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3c24b80157b58b2eb0f673b7768f73da","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5dfc48484ff3623d3ff52edf287e601d22e2b83c1335126c2dbc7ea70ce18136","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.870Z","job_size_bytes":92,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CronEnqueueWorker JID-f0b8197d61b7177431f1b57f: done: 0.08838 sec","job_status":"done","queue_duration_s":1.882578,"scheduling_latency_s":1.882598,"redis_calls":10,"redis_duration_s":0.0016,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000355,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00093,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000315,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055265,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027474,"duration_s":0.08838,"completed_at":"2026-08-15T13:24:29.841Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026892,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.056561] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.842Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c32741b943f13e1a168870d7","created_at":"2026-08-15T13:24:27.883Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b48775e429b0e7b6edcd49723ebb0235","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.908Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c32741b943f13e1a168870d7: start","job_status":"start","queue_duration_s":1.933972,"scheduling_latency_s":1.933992} gitlab # [ 242.070903] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.875Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"c32741b943f13e1a168870d7","created_at":"2026-08-15T13:24:27.883Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b48775e429b0e7b6edcd49723ebb0235","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.908Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-c32741b943f13e1a168870d7: done: 0.032819 sec","job_status":"done","queue_duration_s":1.933972,"scheduling_latency_s":1.933992,"redis_calls":13,"redis_duration_s":0.0020210000000000002,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00032,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000731,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020641,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.009781399999999999,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.032819,"completed_at":"2026-08-15T13:24:29.875Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004446,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 242.117398] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.876Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"f10453958739b9755e803606","created_at":"2026-08-15T13:24:27.949Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7298d2b5a51e4ab910f2891b0542b1be","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.983Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-f10453958739b9755e803606: start","job_status":"start","queue_duration_s":1.893019,"scheduling_latency_s":1.893038} gitlab # [ 242.131520] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.905Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Environments::AutoStopCronWorker","jid":"f10453958739b9755e803606","created_at":"2026-08-15T13:24:27.949Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7298d2b5a51e4ab910f2891b0542b1be","meta.root_caller_id":"Cronjob","meta.feature_category":"continuous_delivery","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:037d6a92006e3beb0e73d758a3f9a2a2a104eaa8b9d3c2b1a2cf37915906a513","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:27.983Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Environments::AutoStopCronWorker JID-f10453958739b9755e803606: done: 0.028441 sec","job_status":"done","queue_duration_s":1.893019,"scheduling_latency_s":1.893038,"redis_calls":12,"redis_duration_s":0.001686,"redis_read_bytes":11,"redis_write_bytes":1368,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000275,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000767,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000644,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":503,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016797,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00046926,"exclusive_lock_hold_duration_s":0.00893066,"duration_s":0.028441,"completed_at":"2026-08-15T13:24:29.905Z","load_balancing_strategy":"primary","db_duration_s":0.003942,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.173557] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.906Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"8c71070333826ac2834b1d6d","created_at":"2026-08-15T13:24:28.005Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"3f60053a2fef9c35058a909631270b27","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.012Z","job_size_bytes":3,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-8c71070333826ac2834b1d6d: start","job_status":"start","queue_duration_s":1.893872,"scheduling_latency_s":1.893891} gitlab # [ 242.187548] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.927Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"8c71070333826ac2834b1d6d","created_at":"2026-08-15T13:24:28.005Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"3f60053a2fef9c35058a909631270b27","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.012Z","job_size_bytes":3,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-8c71070333826ac2834b1d6d: done: 0.020556 sec","job_status":"done","queue_duration_s":1.893872,"scheduling_latency_s":1.893891,"redis_calls":10,"redis_duration_s":0.001477,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000334,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000308,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025214,"duration_s":0.020556,"completed_at":"2026-08-15T13:24:29.926Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000628,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.229530] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"378264102e79e190803fbd27","created_at":"2026-08-15T13:24:28.028Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1f6f66b0b82a6b852ac66b53f1111ae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.053Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-378264102e79e190803fbd27: start","job_status":"start","queue_duration_s":1.874608,"scheduling_latency_s":1.874624} gitlab # [ 242.243317] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.963Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"378264102e79e190803fbd27","created_at":"2026-08-15T13:24:28.028Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d1f6f66b0b82a6b852ac66b53f1111ae","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.053Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-378264102e79e190803fbd27: done: 0.035398 sec","job_status":"done","queue_duration_s":1.874608,"scheduling_latency_s":1.874624,"redis_calls":16,"redis_duration_s":0.0021809999999999998,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001107,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000286,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019729,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002406,"duration_s":0.035398,"completed_at":"2026-08-15T13:24:29.963Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008713,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.286200] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:29.964Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f0c54d848f566c79acb33069","created_at":"2026-08-15T13:24:28.070Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.104Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f0c54d848f566c79acb33069: start","job_status":"start","queue_duration_s":1.860593,"scheduling_latency_s":1.860608} gitlab # [ 242.301039] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.002Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"f0c54d848f566c79acb33069","created_at":"2026-08-15T13:24:28.070Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:24:28.104Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-f0c54d848f566c79acb33069: done: 0.037271 sec","job_status":"done","queue_duration_s":1.860593,"scheduling_latency_s":1.860608,"redis_calls":19,"redis_duration_s":0.002637,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000578,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.00022,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001402,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026506,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002468,"duration_s":0.037271,"completed_at":"2026-08-15T13:24:30.002Z","load_balancing_strategy":"primary","db_duration_s":0.00338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.343995] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eff18437a2e224be9d8dee8a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eff18437a2e224be9d8dee8a: start","job_status":"start","queue_duration_s":0.788578,"scheduling_latency_s":0.788595} gitlab # [ 242.356140] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.027Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eff18437a2e224be9d8dee8a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eff18437a2e224be9d8dee8a: done: 0.023738 sec","job_status":"done","queue_duration_s":0.788578,"scheduling_latency_s":0.788595,"redis_calls":16,"redis_duration_s":0.0023,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000604,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001387,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014683,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024416,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023738,"completed_at":"2026-08-15T13:24:30.026Z","load_balancing_strategy":"primary","db_duration_s":0.000762,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.397379] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.028Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"341337da3c9dfb5d09b21e39","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-341337da3c9dfb5d09b21e39: start","job_status":"start","queue_duration_s":0.813515,"scheduling_latency_s":0.813532} gitlab # [ 242.409710] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.044Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"341337da3c9dfb5d09b21e39","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-341337da3c9dfb5d09b21e39: done: 0.016458 sec","job_status":"done","queue_duration_s":0.813515,"scheduling_latency_s":0.813532,"redis_calls":13,"redis_duration_s":0.00249,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01148,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016458,"completed_at":"2026-08-15T13:24:30.044Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.449162] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.045Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbf0f4cf2d66993048110a1a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbf0f4cf2d66993048110a1a: start","job_status":"start","queue_duration_s":0.831106,"scheduling_latency_s":0.831124} gitlab # [ 242.461483] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.061Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bbf0f4cf2d66993048110a1a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bbf0f4cf2d66993048110a1a: done: 0.016108 sec","job_status":"done","queue_duration_s":0.831106,"scheduling_latency_s":0.831124,"redis_calls":13,"redis_duration_s":0.00167,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011361,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016108,"completed_at":"2026-08-15T13:24:30.061Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.502061] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2994c1a268f105d088009538","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2994c1a268f105d088009538: start","job_status":"start","queue_duration_s":0.848441,"scheduling_latency_s":0.848458} gitlab # [ 242.514639] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.078Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2994c1a268f105d088009538","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2994c1a268f105d088009538: done: 0.015658 sec","job_status":"done","queue_duration_s":0.848441,"scheduling_latency_s":0.848458,"redis_calls":13,"redis_duration_s":0.001676,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00063,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015658,"completed_at":"2026-08-15T13:24:30.078Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.554030] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.080Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf194191a862e0017cc16ec4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf194191a862e0017cc16ec4: start","job_status":"start","queue_duration_s":0.866047,"scheduling_latency_s":0.866064} gitlab # [ 242.566604] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.095Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bf194191a862e0017cc16ec4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bf194191a862e0017cc16ec4: done: 0.015056 sec","job_status":"done","queue_duration_s":0.866047,"scheduling_latency_s":0.866064,"redis_calls":13,"redis_duration_s":0.0015179999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015056,"completed_at":"2026-08-15T13:24:30.095Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.606103] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.097Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f2f93ce9e392f6e6b2c3c06","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f2f93ce9e392f6e6b2c3c06: start","job_status":"start","queue_duration_s":0.882238,"scheduling_latency_s":0.882254} gitlab # [ 242.618692] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5f2f93ce9e392f6e6b2c3c06","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5f2f93ce9e392f6e6b2c3c06: done: 0.015234 sec","job_status":"done","queue_duration_s":0.882238,"scheduling_latency_s":0.882254,"redis_calls":13,"redis_duration_s":0.001546,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010739,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015234,"completed_at":"2026-08-15T13:24:30.112Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.658044] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.113Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"210e72700d0c237cfa921f2b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-210e72700d0c237cfa921f2b: start","job_status":"start","queue_duration_s":0.898606,"scheduling_latency_s":0.898622} gitlab # [ 242.670606] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.129Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"210e72700d0c237cfa921f2b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-210e72700d0c237cfa921f2b: done: 0.015888 sec","job_status":"done","queue_duration_s":0.898606,"scheduling_latency_s":0.898622,"redis_calls":13,"redis_duration_s":0.001586,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001037,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015888,"completed_at":"2026-08-15T13:24:30.129Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.711342] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea20a9f35fcafb17debd2865","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea20a9f35fcafb17debd2865: start","job_status":"start","queue_duration_s":0.915688,"scheduling_latency_s":0.915706} gitlab # [ 242.723951] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea20a9f35fcafb17debd2865","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea20a9f35fcafb17debd2865: done: 0.018274 sec","job_status":"done","queue_duration_s":0.915688,"scheduling_latency_s":0.915706,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018274,"completed_at":"2026-08-15T13:24:30.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.763478] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdc0ed2c68349c2836b830a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdc0ed2c68349c2836b830a: start","job_status":"start","queue_duration_s":0.935083,"scheduling_latency_s":0.935099} gitlab # [ 242.776167] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.165Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4fdc0ed2c68349c2836b830a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4fdc0ed2c68349c2836b830a: done: 0.015932 sec","job_status":"done","queue_duration_s":0.935083,"scheduling_latency_s":0.935099,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015932,"completed_at":"2026-08-15T13:24:30.165Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.815592] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ca83d4396697d79c05927cb","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ca83d4396697d79c05927cb: start","job_status":"start","queue_duration_s":0.952149,"scheduling_latency_s":0.952166} gitlab # [ 242.828377] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.182Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9ca83d4396697d79c05927cb","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9ca83d4396697d79c05927cb: done: 0.015716 sec","job_status":"done","queue_duration_s":0.952149,"scheduling_latency_s":0.952166,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000943,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015716,"completed_at":"2026-08-15T13:24:30.182Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.867155] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.183Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"699415a4c2eb1d8f17bb8513","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-699415a4c2eb1d8f17bb8513: start","job_status":"start","queue_duration_s":0.968985,"scheduling_latency_s":0.969002} gitlab # [ 242.879736] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.198Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"699415a4c2eb1d8f17bb8513","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-699415a4c2eb1d8f17bb8513: done: 0.015208 sec","job_status":"done","queue_duration_s":0.968985,"scheduling_latency_s":0.969002,"redis_calls":13,"redis_duration_s":0.0015630000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015208,"completed_at":"2026-08-15T13:24:30.198Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.920770] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635e9ff8efade897de97f662","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635e9ff8efade897de97f662: start","job_status":"start","queue_duration_s":0.985925,"scheduling_latency_s":0.985942} gitlab # [ 242.933044] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"635e9ff8efade897de97f662","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-635e9ff8efade897de97f662: done: 0.014962 sec","job_status":"done","queue_duration_s":0.985925,"scheduling_latency_s":0.985942,"redis_calls":13,"redis_duration_s":0.0021000000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001146,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01044,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014962,"completed_at":"2026-08-15T13:24:30.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 242.972820] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf018dbd66dc797e46f53c0b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf018dbd66dc797e46f53c0b: start","job_status":"start","queue_duration_s":1.001954,"scheduling_latency_s":1.00197} gitlab # [ 242.985868] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"cf018dbd66dc797e46f53c0b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-cf018dbd66dc797e46f53c0b: done: 0.015795 sec","job_status":"done","queue_duration_s":1.001954,"scheduling_latency_s":1.00197,"redis_calls":13,"redis_duration_s":0.002269,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001725,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015795,"completed_at":"2026-08-15T13:24:30.232Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.025956] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.233Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6023ebb306e551e708facdc","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6023ebb306e551e708facdc: start","job_status":"start","queue_duration_s":1.018845,"scheduling_latency_s":1.01886} gitlab # [ 243.038277] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.249Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6023ebb306e551e708facdc","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6023ebb306e551e708facdc: done: 0.015525 sec","job_status":"done","queue_duration_s":1.018845,"scheduling_latency_s":1.01886,"redis_calls":13,"redis_duration_s":0.002202,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001652,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010604,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015525,"completed_at":"2026-08-15T13:24:30.249Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.078066] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.250Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a8668df2fad5c2ce3f73cc6","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a8668df2fad5c2ce3f73cc6: start","job_status":"start","queue_duration_s":1.035548,"scheduling_latency_s":1.035565} gitlab # [ 243.090676] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0a8668df2fad5c2ce3f73cc6","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0a8668df2fad5c2ce3f73cc6: done: 0.01466 sec","job_status":"done","queue_duration_s":1.035548,"scheduling_latency_s":1.035565,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000517,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01466,"completed_at":"2026-08-15T13:24:30.264Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.131787] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.266Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eee33bae5d29d308b41778a8","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eee33bae5d29d308b41778a8: start","job_status":"start","queue_duration_s":1.051777,"scheduling_latency_s":1.051795} gitlab # [ 243.144510] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.281Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"eee33bae5d29d308b41778a8","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-eee33bae5d29d308b41778a8: done: 0.014588 sec","job_status":"done","queue_duration_s":1.051777,"scheduling_latency_s":1.051795,"redis_calls":13,"redis_duration_s":0.001524,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010189,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014588,"completed_at":"2026-08-15T13:24:30.281Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.183614] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3f16120ea7d552713ee25dc","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3f16120ea7d552713ee25dc: start","job_status":"start","queue_duration_s":1.067502,"scheduling_latency_s":1.067518} gitlab # [ 243.196033] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d3f16120ea7d552713ee25dc","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d3f16120ea7d552713ee25dc: done: 0.014359 sec","job_status":"done","queue_duration_s":1.067502,"scheduling_latency_s":1.067518,"redis_calls":13,"redis_duration_s":0.001529,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014359,"completed_at":"2026-08-15T13:24:30.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.235622] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.297Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07daaa35dafe928e2b1d263c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07daaa35dafe928e2b1d263c: start","job_status":"start","queue_duration_s":1.082952,"scheduling_latency_s":1.082968} gitlab # [ 243.248391] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.312Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07daaa35dafe928e2b1d263c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07daaa35dafe928e2b1d263c: done: 0.014708 sec","job_status":"done","queue_duration_s":1.082952,"scheduling_latency_s":1.082968,"redis_calls":13,"redis_duration_s":0.001528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000541,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014708,"completed_at":"2026-08-15T13:24:30.312Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.287403] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.313Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e0a45e720f1419449df2ba","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e0a45e720f1419449df2ba: start","job_status":"start","queue_duration_s":1.098742,"scheduling_latency_s":1.098757} gitlab # [ 243.300056] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83e0a45e720f1419449df2ba","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83e0a45e720f1419449df2ba: done: 0.01437 sec","job_status":"done","queue_duration_s":1.098742,"scheduling_latency_s":1.098757,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000522,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009776,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01437,"completed_at":"2026-08-15T13:24:30.327Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.340137] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.329Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9c572b7b4ef36fdf6bbb4be","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9c572b7b4ef36fdf6bbb4be: start","job_status":"start","queue_duration_s":1.114348,"scheduling_latency_s":1.114365} gitlab # [ 243.352532] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b9c572b7b4ef36fdf6bbb4be","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b9c572b7b4ef36fdf6bbb4be: done: 0.014711 sec","job_status":"done","queue_duration_s":1.114348,"scheduling_latency_s":1.114365,"redis_calls":13,"redis_duration_s":0.001799,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000838,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010017,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014711,"completed_at":"2026-08-15T13:24:30.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.391506] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.344Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4540070048afb59851a0689b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4540070048afb59851a0689b: start","job_status":"start","queue_duration_s":1.130181,"scheduling_latency_s":1.130198} gitlab # [ 243.404096] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.359Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4540070048afb59851a0689b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4540070048afb59851a0689b: done: 0.01501 sec","job_status":"done","queue_duration_s":1.130181,"scheduling_latency_s":1.130198,"redis_calls":13,"redis_duration_s":0.001601,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001039,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010452,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01501,"completed_at":"2026-08-15T13:24:30.359Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.442948] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a900132bde402b48ddd81d","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a900132bde402b48ddd81d: start","job_status":"start","queue_duration_s":1.146356,"scheduling_latency_s":1.146374} gitlab # [ 243.455507] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e0a900132bde402b48ddd81d","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e0a900132bde402b48ddd81d: done: 0.014997 sec","job_status":"done","queue_duration_s":1.146356,"scheduling_latency_s":1.146374,"redis_calls":13,"redis_duration_s":0.001597,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014997,"completed_at":"2026-08-15T13:24:30.376Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.495023] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.377Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d59f03efd9924c10ee38c4cd","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d59f03efd9924c10ee38c4cd: start","job_status":"start","queue_duration_s":1.162763,"scheduling_latency_s":1.162779} gitlab # [ 243.507484] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.392Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d59f03efd9924c10ee38c4cd","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d59f03efd9924c10ee38c4cd: done: 0.014715 sec","job_status":"done","queue_duration_s":1.162763,"scheduling_latency_s":1.162779,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010108,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014715,"completed_at":"2026-08-15T13:24:30.392Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.547651] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.393Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185d3a4cb53e3825bc85746c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185d3a4cb53e3825bc85746c: start","job_status":"start","queue_duration_s":1.178794,"scheduling_latency_s":1.17881} gitlab # [ 243.560169] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.407Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"185d3a4cb53e3825bc85746c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-185d3a4cb53e3825bc85746c: done: 0.014457 sec","job_status":"done","queue_duration_s":1.178794,"scheduling_latency_s":1.17881,"redis_calls":13,"redis_duration_s":0.001493,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010042,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014457,"completed_at":"2026-08-15T13:24:30.407Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.598854] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.409Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f686503005eb1b4f112981b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f686503005eb1b4f112981b: start","job_status":"start","queue_duration_s":1.194413,"scheduling_latency_s":1.194429} gitlab # [ 243.611379] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.423Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2f686503005eb1b4f112981b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2f686503005eb1b4f112981b: done: 0.014602 sec","job_status":"done","queue_duration_s":1.194413,"scheduling_latency_s":1.194429,"redis_calls":13,"redis_duration_s":0.001466,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000938,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009941,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014602,"completed_at":"2026-08-15T13:24:30.423Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.650294] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"144843efc82a17dfdaf5a60e","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-144843efc82a17dfdaf5a60e: start","job_status":"start","queue_duration_s":1.210141,"scheduling_latency_s":1.210158} gitlab # [ 243.662704] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.440Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"144843efc82a17dfdaf5a60e","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-144843efc82a17dfdaf5a60e: done: 0.015166 sec","job_status":"done","queue_duration_s":1.210141,"scheduling_latency_s":1.210158,"redis_calls":13,"redis_duration_s":0.0015660000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015166,"completed_at":"2026-08-15T13:24:30.439Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.702045] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.441Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9844afbf42aea268faad64f","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9844afbf42aea268faad64f: start","job_status":"start","queue_duration_s":1.226715,"scheduling_latency_s":1.22673} gitlab # [ 243.714503] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.455Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c9844afbf42aea268faad64f","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c9844afbf42aea268faad64f: done: 0.014375 sec","job_status":"done","queue_duration_s":1.226715,"scheduling_latency_s":1.22673,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009928,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014375,"completed_at":"2026-08-15T13:24:30.455Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.754723] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.456Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0ae1ccec36315c77f96864f","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0ae1ccec36315c77f96864f: start","job_status":"start","queue_duration_s":1.242236,"scheduling_latency_s":1.242251} gitlab # [ 243.767248] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.471Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0ae1ccec36315c77f96864f","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0ae1ccec36315c77f96864f: done: 0.014588 sec","job_status":"done","queue_duration_s":1.242236,"scheduling_latency_s":1.242251,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014588,"completed_at":"2026-08-15T13:24:30.471Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.806444] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.472Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747c016c34c892af6fe00313","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747c016c34c892af6fe00313: start","job_status":"start","queue_duration_s":1.258018,"scheduling_latency_s":1.258035} gitlab # [ 243.818983] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.487Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"747c016c34c892af6fe00313","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-747c016c34c892af6fe00313: done: 0.014689 sec","job_status":"done","queue_duration_s":1.258018,"scheduling_latency_s":1.258035,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014689,"completed_at":"2026-08-15T13:24:30.487Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.858316] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1115d14c7c076d9a58996b83","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1115d14c7c076d9a58996b83: start","job_status":"start","queue_duration_s":1.273853,"scheduling_latency_s":1.273869} gitlab # [ 243.870938] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1115d14c7c076d9a58996b83","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1115d14c7c076d9a58996b83: done: 0.014535 sec","job_status":"done","queue_duration_s":1.273853,"scheduling_latency_s":1.273869,"redis_calls":13,"redis_duration_s":0.001456,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009792,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014535,"completed_at":"2026-08-15T13:24:30.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.910162] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6fce569a3754946274dc410","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6fce569a3754946274dc410: start","job_status":"start","queue_duration_s":1.289914,"scheduling_latency_s":1.28993} gitlab # [ 243.922690] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6fce569a3754946274dc410","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6fce569a3754946274dc410: done: 0.014517 sec","job_status":"done","queue_duration_s":1.289914,"scheduling_latency_s":1.28993,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010105,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014517,"completed_at":"2026-08-15T13:24:30.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 243.963082] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec76d3a1cf67fd355dcca99","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec76d3a1cf67fd355dcca99: start","job_status":"start","queue_duration_s":1.305559,"scheduling_latency_s":1.305576} gitlab # [ 243.975708] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.534Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ec76d3a1cf67fd355dcca99","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ec76d3a1cf67fd355dcca99: done: 0.014647 sec","job_status":"done","queue_duration_s":1.305559,"scheduling_latency_s":1.305576,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010177,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014647,"completed_at":"2026-08-15T13:24:30.534Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.015120] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47d3551ae1b2308716ed4b0","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47d3551ae1b2308716ed4b0: start","job_status":"start","queue_duration_s":1.321682,"scheduling_latency_s":1.3217} gitlab # [ 244.027839] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f47d3551ae1b2308716ed4b0","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f47d3551ae1b2308716ed4b0: done: 0.014782 sec","job_status":"done","queue_duration_s":1.321682,"scheduling_latency_s":1.3217,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014782,"completed_at":"2026-08-15T13:24:30.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.067493] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e91e3a309b6176b021e147ac","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e91e3a309b6176b021e147ac: start","job_status":"start","queue_duration_s":1.337542,"scheduling_latency_s":1.337558} gitlab # [ 244.080226] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.566Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e91e3a309b6176b021e147ac","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e91e3a309b6176b021e147ac: done: 0.014368 sec","job_status":"done","queue_duration_s":1.337542,"scheduling_latency_s":1.337558,"redis_calls":13,"redis_duration_s":0.001446,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009663,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014368,"completed_at":"2026-08-15T13:24:30.566Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.119812] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69fe241e6a06fc4c1f277a2","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69fe241e6a06fc4c1f277a2: start","job_status":"start","queue_duration_s":1.353329,"scheduling_latency_s":1.353344} gitlab # [ 244.133861] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c69fe241e6a06fc4c1f277a2","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c69fe241e6a06fc4c1f277a2: done: 0.014636 sec","job_status":"done","queue_duration_s":1.353329,"scheduling_latency_s":1.353344,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000941,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014636,"completed_at":"2026-08-15T13:24:30.582Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.172679] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acd60104efb66d5af39ee3d","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acd60104efb66d5af39ee3d: start","job_status":"start","queue_duration_s":1.369189,"scheduling_latency_s":1.369206} gitlab # [ 244.185008] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.599Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acd60104efb66d5af39ee3d","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acd60104efb66d5af39ee3d: done: 0.015316 sec","job_status":"done","queue_duration_s":1.369189,"scheduling_latency_s":1.369206,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010831,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015316,"completed_at":"2026-08-15T13:24:30.599Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.223834] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c332fd8871ad6bfb2fc83e4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c332fd8871ad6bfb2fc83e4: start","job_status":"start","queue_duration_s":1.385608,"scheduling_latency_s":1.385625} gitlab # [ 244.236156] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2c332fd8871ad6bfb2fc83e4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2c332fd8871ad6bfb2fc83e4: done: 0.0142 sec","job_status":"done","queue_duration_s":1.385608,"scheduling_latency_s":1.385625,"redis_calls":13,"redis_duration_s":0.001452,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000529,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000923,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009753,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0142,"completed_at":"2026-08-15T13:24:30.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.274955] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86b3c7cf1163a5c1c03028b3","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86b3c7cf1163a5c1c03028b3: start","job_status":"start","queue_duration_s":1.400969,"scheduling_latency_s":1.400986} gitlab # [ 244.287538] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"86b3c7cf1163a5c1c03028b3","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-86b3c7cf1163a5c1c03028b3: done: 0.014532 sec","job_status":"done","queue_duration_s":1.400969,"scheduling_latency_s":1.400986,"redis_calls":13,"redis_duration_s":0.001474,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00982,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014532,"completed_at":"2026-08-15T13:24:30.630Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.329336] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.631Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d1ce8d6b9ada260d0215ca","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d1ce8d6b9ada260d0215ca: start","job_status":"start","queue_duration_s":1.41703,"scheduling_latency_s":1.417046} gitlab # [ 244.342770] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.648Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97d1ce8d6b9ada260d0215ca","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97d1ce8d6b9ada260d0215ca: done: 0.016658 sec","job_status":"done","queue_duration_s":1.41703,"scheduling_latency_s":1.417046,"redis_calls":13,"redis_duration_s":0.001646,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000597,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001049,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011913,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016658,"completed_at":"2026-08-15T13:24:30.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.381910] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0431842e065fa577e281723","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0431842e065fa577e281723: start","job_status":"start","queue_duration_s":1.435404,"scheduling_latency_s":1.435421} gitlab # [ 244.394413] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f0431842e065fa577e281723","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f0431842e065fa577e281723: done: 0.016285 sec","job_status":"done","queue_duration_s":1.435404,"scheduling_latency_s":1.435421,"redis_calls":13,"redis_duration_s":0.001579,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000594,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011387,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016285,"completed_at":"2026-08-15T13:24:30.666Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.434188] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.668Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b001cdc3cd0acab278b4544","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b001cdc3cd0acab278b4544: start","job_status":"start","queue_duration_s":1.453336,"scheduling_latency_s":1.453354} gitlab # [ 244.446635] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.688Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2b001cdc3cd0acab278b4544","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2b001cdc3cd0acab278b4544: done: 0.020085 sec","job_status":"done","queue_duration_s":1.453336,"scheduling_latency_s":1.453354,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001008,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011268,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020085,"completed_at":"2026-08-15T13:24:30.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.485564] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e99359ef4a9390445e179a82","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e99359ef4a9390445e179a82: start","job_status":"start","queue_duration_s":1.474603,"scheduling_latency_s":1.474619} gitlab # [ 244.497720] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e99359ef4a9390445e179a82","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e99359ef4a9390445e179a82: done: 0.016401 sec","job_status":"done","queue_duration_s":1.474603,"scheduling_latency_s":1.474619,"redis_calls":13,"redis_duration_s":0.002089,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001508,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016401,"completed_at":"2026-08-15T13:24:30.705Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.537629] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.707Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f137bc7f1314030e9ae304b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f137bc7f1314030e9ae304b: start","job_status":"start","queue_duration_s":1.4923,"scheduling_latency_s":1.492318} gitlab # [ 244.549958] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.723Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f137bc7f1314030e9ae304b","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f137bc7f1314030e9ae304b: done: 0.016139 sec","job_status":"done","queue_duration_s":1.4923,"scheduling_latency_s":1.492318,"redis_calls":13,"redis_duration_s":0.00194,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000533,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001407,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011153,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016139,"completed_at":"2026-08-15T13:24:30.723Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.588940] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.724Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54ea7d33c1dbe66700a21a8c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54ea7d33c1dbe66700a21a8c: start","job_status":"start","queue_duration_s":1.509597,"scheduling_latency_s":1.509614} gitlab # [ 244.601284] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.741Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"54ea7d33c1dbe66700a21a8c","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-54ea7d33c1dbe66700a21a8c: done: 0.016783 sec","job_status":"done","queue_duration_s":1.509597,"scheduling_latency_s":1.509614,"redis_calls":13,"redis_duration_s":0.0018260000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000595,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001231,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011087,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016783,"completed_at":"2026-08-15T13:24:30.741Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.640658] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.742Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f7f36e10d524a25d2d748c0","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f7f36e10d524a25d2d748c0: start","job_status":"start","queue_duration_s":1.527563,"scheduling_latency_s":1.527578} gitlab # [ 244.652934] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.759Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7f7f36e10d524a25d2d748c0","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7f7f36e10d524a25d2d748c0: done: 0.016889 sec","job_status":"done","queue_duration_s":1.527563,"scheduling_latency_s":1.527578,"redis_calls":13,"redis_duration_s":0.0016049999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001004,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016889,"completed_at":"2026-08-15T13:24:30.759Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.692169] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.760Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f132deaafa7b1c78031a364","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f132deaafa7b1c78031a364: start","job_status":"start","queue_duration_s":1.545629,"scheduling_latency_s":1.545647} gitlab # [ 244.704435] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.777Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3f132deaafa7b1c78031a364","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3f132deaafa7b1c78031a364: done: 0.01675 sec","job_status":"done","queue_duration_s":1.545629,"scheduling_latency_s":1.545647,"redis_calls":13,"redis_duration_s":0.0017330000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000623,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00111,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01675,"completed_at":"2026-08-15T13:24:30.777Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.744869] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.778Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4038b07f33a71c9fd11d9c4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4038b07f33a71c9fd11d9c4: start","job_status":"start","queue_duration_s":1.564166,"scheduling_latency_s":1.564186} gitlab # [ 244.756890] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.796Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4038b07f33a71c9fd11d9c4","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4038b07f33a71c9fd11d9c4: done: 0.0172 sec","job_status":"done","queue_duration_s":1.564166,"scheduling_latency_s":1.564186,"redis_calls":13,"redis_duration_s":0.001629,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001064,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011601,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0172,"completed_at":"2026-08-15T13:24:30.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.796073] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3a0259323c43da82f4c83a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3a0259323c43da82f4c83a: start","job_status":"start","queue_duration_s":1.582924,"scheduling_latency_s":1.58294} gitlab # [ 244.808325] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1f3a0259323c43da82f4c83a","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1f3a0259323c43da82f4c83a: done: 0.015852 sec","job_status":"done","queue_duration_s":1.582924,"scheduling_latency_s":1.58294,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011427,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015852,"completed_at":"2026-08-15T13:24:30.813Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.847505] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77ac648f772a09fa147ad1ca","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77ac648f772a09fa147ad1ca: start","job_status":"start","queue_duration_s":1.600473,"scheduling_latency_s":1.60049} gitlab # [ 244.860038] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.831Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"77ac648f772a09fa147ad1ca","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-77ac648f772a09fa147ad1ca: done: 0.016311 sec","job_status":"done","queue_duration_s":1.600473,"scheduling_latency_s":1.60049,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011838,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016311,"completed_at":"2026-08-15T13:24:30.831Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.899114] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc05e7cc1e8c4369fe0c8756","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc05e7cc1e8c4369fe0c8756: start","job_status":"start","queue_duration_s":1.618371,"scheduling_latency_s":1.618388} gitlab # [ 244.911454] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.849Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc05e7cc1e8c4369fe0c8756","created_at":"2026-08-15T13:24:29.090Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-1307d9d01e7d4b2f","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"4cc9a22d6940c7762d591af3ff17c16b","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.214Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc05e7cc1e8c4369fe0c8756: done: 0.016455 sec","job_status":"done","queue_duration_s":1.618371,"scheduling_latency_s":1.618388,"redis_calls":13,"redis_duration_s":0.001572,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011702,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016455,"completed_at":"2026-08-15T13:24:30.849Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 244.951712] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.851Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55b02fc6cb13345949c041c6","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55b02fc6cb13345949c041c6: start","job_status":"start","queue_duration_s":0.856921,"scheduling_latency_s":0.856939} gitlab # [ 244.964249] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.869Z","message":"No existing merge request to be cleaned up."} gitlab # [ 244.965874] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.886Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"55b02fc6cb13345949c041c6","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-55b02fc6cb13345949c041c6: done: 0.035098 sec","job_status":"done","queue_duration_s":0.856921,"scheduling_latency_s":0.856939,"redis_calls":13,"redis_duration_s":0.001784,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000316,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000638,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00083,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002528,"duration_s":0.035098,"completed_at":"2026-08-15T13:24:30.886Z","load_balancing_strategy":"primary","db_duration_s":0.006241,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.005439] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.888Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2876114d6fa73bf5218af7c1","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2876114d6fa73bf5218af7c1: start","job_status":"start","queue_duration_s":0.893684,"scheduling_latency_s":0.893701} gitlab # [ 245.017390] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.902Z","message":"No existing merge request to be cleaned up."} gitlab # [ 245.018990] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.911Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"2876114d6fa73bf5218af7c1","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-2876114d6fa73bf5218af7c1: done: 0.023221 sec","job_status":"done","queue_duration_s":0.893684,"scheduling_latency_s":0.893701,"redis_calls":10,"redis_duration_s":0.001562,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016089,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023221,"completed_at":"2026-08-15T13:24:30.911Z","load_balancing_strategy":"primary","db_duration_s":0.002843,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.057193] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.912Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d58780d6900f468e9b8c0724","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d58780d6900f468e9b8c0724: start","job_status":"start","queue_duration_s":0.918022,"scheduling_latency_s":0.918038} gitlab # [ 245.069447] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.924Z","message":"No existing merge request to be cleaned up."} gitlab # [ 245.071093] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.932Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d58780d6900f468e9b8c0724","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d58780d6900f468e9b8c0724: done: 0.020521 sec","job_status":"done","queue_duration_s":0.918022,"scheduling_latency_s":0.918038,"redis_calls":10,"redis_duration_s":0.001774,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00099,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000784,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020521,"completed_at":"2026-08-15T13:24:30.932Z","load_balancing_strategy":"primary","db_duration_s":0.002512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 245.109714] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.934Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dbb70de2e833e411580be13a","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dbb70de2e833e411580be13a: start","job_status":"start","queue_duration_s":0.939652,"scheduling_latency_s":0.939669} gitlab # [ 245.121918] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.946Z","message":"No existing merge request to be cleaned up."} gitlab # [ 245.123497] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:24:30.955Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"dbb70de2e833e411580be13a","created_at":"2026-08-15T13:24:29.981Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-75fb8229428a4aea","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"bd23b90c943a8b917434caf99b1e6b67","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:24:29.994Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-dbb70de2e833e411580be13a: done: 0.021418 sec","job_status":"done","queue_duration_s":0.939652,"scheduling_latency_s":0.939669,"redis_calls":10,"redis_duration_s":0.001702,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001058,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014505,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021418,"completed_at":"2026-08-15T13:24:30.955Z","load_balancing_strategy":"primary","db_duration_s":0.002903,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.101452] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:03.930Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e6a6b62e3107706ce748c2b6","created_at":"2026-08-15T13:25:03.912Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3943914ac50b1daaf8a5d7563ef26cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.926Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e6a6b62e3107706ce748c2b6: start","job_status":"start","queue_duration_s":0.003869,"scheduling_latency_s":0.003925} gitlab # [ 276.400207] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.229Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"e6a6b62e3107706ce748c2b6","created_at":"2026-08-15T13:25:03.912Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3943914ac50b1daaf8a5d7563ef26cf7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.926Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-e6a6b62e3107706ce748c2b6: done: 0.299293 sec","job_status":"done","queue_duration_s":0.003869,"scheduling_latency_s":0.003925,"redis_calls":12,"redis_duration_s":0.034669,"redis_read_bytes":143,"redis_write_bytes":1234,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.01059,"redis_feature_flag_read_bytes":132,"redis_feature_flag_write_bytes":68,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.014816,"redis_queues_metadata_read_bytes":8,"redis_queues_metadata_write_bytes":555,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.009263,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.06,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.06,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.066472,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00791242,"exclusive_lock_hold_duration_s":0.14864002,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.299293,"completed_at":"2026-08-15T13:25:04.229Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.105124,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.497220] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.261Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8a161ffc120f2f465be6fac3","created_at":"2026-08-15T13:25:03.943Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8e6bab2e66523d8a82063179880044a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.948Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8a161ffc120f2f465be6fac3: start","job_status":"start","queue_duration_s":0.312587,"scheduling_latency_s":0.312635} gitlab # [ 276.518815] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.347Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 276.549006] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"8a161ffc120f2f465be6fac3","created_at":"2026-08-15T13:25:03.943Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8e6bab2e66523d8a82063179880044a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.948Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-8a161ffc120f2f465be6fac3: done: 0.118372 sec","job_status":"done","queue_duration_s":0.312587,"scheduling_latency_s":0.312635,"redis_calls":7,"redis_duration_s":0.00251,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002125,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000385,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.118372,"completed_at":"2026-08-15T13:25:04.379Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.621810] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.395Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e756ac4b5d40f409401e6aff","created_at":"2026-08-15T13:25:03.973Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f2f82fd5a3554523e2694c8929fac01","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.980Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e756ac4b5d40f409401e6aff: start","job_status":"start","queue_duration_s":0.414989,"scheduling_latency_s":0.415011} gitlab # [ 276.638666] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.428Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"e756ac4b5d40f409401e6aff","created_at":"2026-08-15T13:25:03.973Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f2f82fd5a3554523e2694c8929fac01","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:03.980Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-e756ac4b5d40f409401e6aff: done: 0.032949 sec","job_status":"done","queue_duration_s":0.414989,"scheduling_latency_s":0.415011,"redis_calls":8,"redis_duration_s":0.001803,"redis_read_bytes":133,"redis_write_bytes":730,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000169,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":49,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014918,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032949,"completed_at":"2026-08-15T13:25:04.428Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002185,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.681644] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.430Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1ff611bd1b0b9b671fe332c7","created_at":"2026-08-15T13:25:03.995Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46db1215fb40662d64f513376f3805c1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.000Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1ff611bd1b0b9b671fe332c7: start","job_status":"start","queue_duration_s":0.429764,"scheduling_latency_s":0.429788} gitlab # [ 276.694964] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.506Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1ff611bd1b0b9b671fe332c7","created_at":"2026-08-15T13:25:03.995Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"46db1215fb40662d64f513376f3805c1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.000Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1ff611bd1b0b9b671fe332c7: done: 0.076179 sec","job_status":"done","queue_duration_s":0.429764,"scheduling_latency_s":0.429788,"redis_calls":9,"redis_duration_s":0.002316,"redis_read_bytes":7,"redis_write_bytes":1072,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001088,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001228,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.068013,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00087508,"exclusive_lock_hold_duration_s":0.00497846,"duration_s":0.076179,"completed_at":"2026-08-15T13:25:04.506Z","load_balancing_strategy":"primary","db_duration_s":0.001492,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.734872] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.507Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"83eff809e2f0ebe2ef037841","created_at":"2026-08-15T13:25:04.028Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"948b321617d35d9ec2b3ee818a4e755d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.075Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-83eff809e2f0ebe2ef037841: start","job_status":"start","queue_duration_s":0.432117,"scheduling_latency_s":0.432132} gitlab # [ 276.749574] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.527Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::StaleRequestsCleanupCronWorker","jid":"83eff809e2f0ebe2ef037841","created_at":"2026-08-15T13:25:04.028Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"948b321617d35d9ec2b3ee818a4e755d","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:73d233265a8deedc9c2e88dc2f4303db4adf3b844f30d267f04d632004c6c278","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.075Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::StaleRequestsCleanupCronWorker JID-83eff809e2f0ebe2ef037841: done: 0.019278 sec","job_status":"done","queue_duration_s":0.432117,"scheduling_latency_s":0.432132,"redis_calls":11,"redis_duration_s":0.002608,"redis_read_bytes":9,"redis_write_bytes":1174,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000465,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001347,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000313,"redis_rate_limiting_write_bytes":51,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000483,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01166,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00035724,"extra.cells_stale_requests_cleanup_cron_worker.removed_count":0,"duration_s":0.019278,"completed_at":"2026-08-15T13:25:04.527Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000342,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.794990] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.528Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b1568892463114a102b44d40","created_at":"2026-08-15T13:25:04.119Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1374494ed996e569a3718441be59cdb8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.126Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b1568892463114a102b44d40: start","job_status":"start","queue_duration_s":0.402242,"scheduling_latency_s":0.402258} gitlab # [ 276.809279] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.549Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"b1568892463114a102b44d40","created_at":"2026-08-15T13:25:04.119Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1374494ed996e569a3718441be59cdb8","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.126Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-b1568892463114a102b44d40: done: 0.021415 sec","job_status":"done","queue_duration_s":0.402242,"scheduling_latency_s":0.402258,"redis_calls":7,"redis_duration_s":0.001691,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001225,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000466,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009235,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.021415,"completed_at":"2026-08-15T13:25:04.549Z","load_balancing_strategy":"primary","db_duration_s":0.006696,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 276.850391] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.552Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"10bc8fe2877af16c9a394a82","created_at":"2026-08-15T13:25:04.131Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de9efa2a6a9c823276018ece0b680c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.136Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-10bc8fe2877af16c9a394a82: start","job_status":"start","queue_duration_s":0.415612,"scheduling_latency_s":0.415626} gitlab # [ 276.951524] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.781Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"10bc8fe2877af16c9a394a82","created_at":"2026-08-15T13:25:04.131Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2de9efa2a6a9c823276018ece0b680c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.136Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-10bc8fe2877af16c9a394a82: done: 0.229743 sec","job_status":"done","queue_duration_s":0.415612,"scheduling_latency_s":0.415626,"redis_calls":9,"redis_duration_s":0.002532,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.029,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.196005,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057622,"exclusive_lock_hold_duration_s":0.21488516,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.229743,"completed_at":"2026-08-15T13:25:04.781Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.043707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.028135] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.783Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ce9ef2af6042815515cc189d","created_at":"2026-08-15T13:25:04.146Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9da73d1a2cad3e255be856a30866115d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.153Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ce9ef2af6042815515cc189d: start","job_status":"start","queue_duration_s":0.630156,"scheduling_latency_s":0.630173} gitlab # [ 277.046495] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.791Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 277.050912] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.797Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"ce9ef2af6042815515cc189d","created_at":"2026-08-15T13:25:04.146Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9da73d1a2cad3e255be856a30866115d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.153Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-ce9ef2af6042815515cc189d: done: 0.013765 sec","job_status":"done","queue_duration_s":0.630156,"scheduling_latency_s":0.630173,"redis_calls":7,"redis_duration_s":0.001866,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001375,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000491,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.007717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013765,"completed_at":"2026-08-15T13:25:04.797Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.092950] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:04.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bc13e91b7682da418d50915c","created_at":"2026-08-15T13:25:04.192Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6fc3313f5c11c5be2dc787340b1a4b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.200Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bc13e91b7682da418d50915c: start","job_status":"start","queue_duration_s":0.599125,"scheduling_latency_s":0.599142} gitlab # [ 277.306165] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.134Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bc13e91b7682da418d50915c","created_at":"2026-08-15T13:25:04.192Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9f6fc3313f5c11c5be2dc787340b1a4b","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.200Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bc13e91b7682da418d50915c: done: 0.335239 sec","job_status":"done","queue_duration_s":0.599125,"scheduling_latency_s":0.599142,"redis_calls":562,"redis_duration_s":0.07455400000000001,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001154,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.0734,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.300371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.335239,"completed_at":"2026-08-15T13:25:05.134Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.399370] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5764a80fa757f25d23cdcdb6","created_at":"2026-08-15T13:25:04.220Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f29bfb2371c22b3339d169f81f994301","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.225Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5764a80fa757f25d23cdcdb6: start","job_status":"start","queue_duration_s":0.911638,"scheduling_latency_s":0.911658} gitlab # [ 277.420757] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"5764a80fa757f25d23cdcdb6","created_at":"2026-08-15T13:25:04.220Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f29bfb2371c22b3339d169f81f994301","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.225Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-5764a80fa757f25d23cdcdb6: done: 0.01874 sec","job_status":"done","queue_duration_s":0.911638,"scheduling_latency_s":0.911658,"redis_calls":9,"redis_duration_s":0.002158,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010679,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00051428,"exclusive_lock_hold_duration_s":0.00453568,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.01874,"completed_at":"2026-08-15T13:25:05.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002676,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.473780] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b59fb78b017e03956d524b39","created_at":"2026-08-15T13:25:04.238Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0135ced054934a08b94cd2abdfbdf58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.244Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b59fb78b017e03956d524b39: start","job_status":"start","queue_duration_s":0.912681,"scheduling_latency_s":0.912696} gitlab # [ 277.488872] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.171Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"b59fb78b017e03956d524b39","created_at":"2026-08-15T13:25:04.238Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0135ced054934a08b94cd2abdfbdf58a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.244Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-b59fb78b017e03956d524b39: done: 0.013909 sec","job_status":"done","queue_duration_s":0.912681,"scheduling_latency_s":0.912696,"redis_calls":8,"redis_duration_s":0.0019089999999999999,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.000236,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.0012,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000473,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013909,"completed_at":"2026-08-15T13:25:05.170Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.531465] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.172Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9c921ef5114593fd7a532b1a","created_at":"2026-08-15T13:25:04.251Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9c921ef5114593fd7a532b1a: start","job_status":"start","queue_duration_s":0.916177,"scheduling_latency_s":0.916191} gitlab # [ 277.545237] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.263Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"9c921ef5114593fd7a532b1a","created_at":"2026-08-15T13:25:04.251Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-9c921ef5114593fd7a532b1a: done: 0.091539 sec","job_status":"done","queue_duration_s":0.916177,"scheduling_latency_s":0.916191,"redis_calls":60,"redis_duration_s":0.015582,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.000952,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013483,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001147,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.078152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.091539,"completed_at":"2026-08-15T13:25:05.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.586745] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.265Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3415e9178fd9cc7cc2870f0f","created_at":"2026-08-15T13:25:04.260Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"715584a55abb4ecbeb4db387b5c176b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:25:04.266Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3415e9178fd9cc7cc2870f0f: start","job_status":"start","queue_duration_s":0.999329,"scheduling_latency_s":0.999344} gitlab # [ 277.601214] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3415e9178fd9cc7cc2870f0f","created_at":"2026-08-15T13:25:04.260Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"715584a55abb4ecbeb4db387b5c176b3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:25:04.266Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3415e9178fd9cc7cc2870f0f: done: 0.023951 sec","job_status":"done","queue_duration_s":0.999329,"scheduling_latency_s":0.999344,"redis_calls":7,"redis_duration_s":0.001894,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001386,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000508,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014916,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023951,"completed_at":"2026-08-15T13:25:05.289Z","load_balancing_strategy":"primary","db_duration_s":0.004142,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.640647] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b7c2fe205495ea41b50e8df7","created_at":"2026-08-15T13:25:04.275Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f28c251f657f3e4a9564c7184800b7b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.279Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b7c2fe205495ea41b50e8df7: start","job_status":"start","queue_duration_s":1.010817,"scheduling_latency_s":1.010833} gitlab # [ 277.655302] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.353Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"b7c2fe205495ea41b50e8df7","created_at":"2026-08-15T13:25:04.275Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f28c251f657f3e4a9564c7184800b7b6","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.279Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-b7c2fe205495ea41b50e8df7: done: 0.062523 sec","job_status":"done","queue_duration_s":1.010817,"scheduling_latency_s":1.010833,"redis_calls":9,"redis_duration_s":0.0028399999999999996,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001419,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001421,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.037,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019441,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00076054,"exclusive_lock_hold_duration_s":0.02628926,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.062523,"completed_at":"2026-08-15T13:25:05.353Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.038381,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.698863] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.355Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3c5a5d05e64b0b6685516b10","created_at":"2026-08-15T13:25:04.289Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7b99dd5f0e26627fc3d71820b3dfc38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.292Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3c5a5d05e64b0b6685516b10: start","job_status":"start","queue_duration_s":1.062511,"scheduling_latency_s":1.062531} gitlab # [ 277.713300] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.364Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 277.716453] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"3c5a5d05e64b0b6685516b10","created_at":"2026-08-15T13:25:04.289Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e7b99dd5f0e26627fc3d71820b3dfc38","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.292Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-3c5a5d05e64b0b6685516b10: done: 0.017392 sec","job_status":"done","queue_duration_s":1.062511,"scheduling_latency_s":1.062531,"redis_calls":7,"redis_duration_s":0.00328,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002816,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009728,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017392,"completed_at":"2026-08-15T13:25:05.372Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.757534] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.374Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"38461d3e37fade86a0fc03e6","created_at":"2026-08-15T13:25:04.313Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0330fa3c51f58205b9b750c51a8f7479","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:25:04.315Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-38461d3e37fade86a0fc03e6: start","job_status":"start","queue_duration_s":1.058763,"scheduling_latency_s":1.058782} gitlab # [ 277.770994] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.383Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 277.774415] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.389Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"38461d3e37fade86a0fc03e6","created_at":"2026-08-15T13:25:04.313Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0330fa3c51f58205b9b750c51a8f7479","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:25:04.315Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-38461d3e37fade86a0fc03e6: done: 0.015 sec","job_status":"done","queue_duration_s":1.058763,"scheduling_latency_s":1.058782,"redis_calls":7,"redis_duration_s":0.0013390000000000001,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009476,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015,"completed_at":"2026-08-15T13:25:05.389Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.815025] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.391Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c1537bbb194d6c9f1423d798","created_at":"2026-08-15T13:25:04.320Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbffa5387d6bf4cfc757e2e106930133","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.322Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c1537bbb194d6c9f1423d798: start","job_status":"start","queue_duration_s":1.0683,"scheduling_latency_s":1.068316} gitlab # [ 277.829778] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"c1537bbb194d6c9f1423d798","created_at":"2026-08-15T13:25:04.320Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"dbffa5387d6bf4cfc757e2e106930133","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.322Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-c1537bbb194d6c9f1423d798: done: 0.0132 sec","job_status":"done","queue_duration_s":1.0683,"scheduling_latency_s":1.068316,"redis_calls":7,"redis_duration_s":0.001395,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001081,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008841,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.0132,"completed_at":"2026-08-15T13:25:05.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.870252] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.405Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3834ec919f15177cced55882","created_at":"2026-08-15T13:25:04.331Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de5b69a4dac02d88b3a249b5a8f03e5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.335Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3834ec919f15177cced55882: start","job_status":"start","queue_duration_s":1.069796,"scheduling_latency_s":1.069812} gitlab # [ 277.884805] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.412Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 277.888299] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.418Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"3834ec919f15177cced55882","created_at":"2026-08-15T13:25:04.331Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de5b69a4dac02d88b3a249b5a8f03e5e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.335Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-3834ec919f15177cced55882: done: 0.012924 sec","job_status":"done","queue_duration_s":1.069796,"scheduling_latency_s":1.069812,"redis_calls":7,"redis_duration_s":0.001529,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000738,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000791,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008157,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012924,"completed_at":"2026-08-15T13:25:05.418Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.928787] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.419Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"6ce0f7b561af7839c792da69","created_at":"2026-08-15T13:25:04.340Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2c110ead3e6b200ab9d5a39121319fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.356Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-6ce0f7b561af7839c792da69: start","job_status":"start","queue_duration_s":1.063226,"scheduling_latency_s":1.063243} gitlab # [ 277.942098] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.452Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ContainerRegistry::CleanupWorker","jid":"6ce0f7b561af7839c792da69","created_at":"2026-08-15T13:25:04.340Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2c110ead3e6b200ab9d5a39121319fc","meta.root_caller_id":"Cronjob","meta.feature_category":"container_registry","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:1c3f6896f5901f854330d52bd75eff7c46411059671aef387767811adc368565","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.356Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ContainerRegistry::CleanupWorker JID-6ce0f7b561af7839c792da69: done: 0.033141 sec","job_status":"done","queue_duration_s":1.063226,"scheduling_latency_s":1.063243,"redis_calls":10,"redis_duration_s":0.001424,"redis_read_bytes":9,"redis_write_bytes":1071,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000289,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000821,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000314,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016661,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026272,"duration_s":0.033141,"completed_at":"2026-08-15T13:25:05.452Z","load_balancing_strategy":"primary","db_duration_s":0.009727,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 277.983240] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f8861ade8a55b9c17306a689","created_at":"2026-08-15T13:25:04.373Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36dc3df87576753cf64c2643aecc7e0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:25:04.377Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f8861ade8a55b9c17306a689: start","job_status":"start","queue_duration_s":1.076641,"scheduling_latency_s":1.076657} gitlab # [ 277.996874] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.461Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 278.000497] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.466Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"f8861ade8a55b9c17306a689","created_at":"2026-08-15T13:25:04.373Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"36dc3df87576753cf64c2643aecc7e0a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:25:04.377Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-f8861ade8a55b9c17306a689: done: 0.012589 sec","job_status":"done","queue_duration_s":1.076641,"scheduling_latency_s":1.076657,"redis_calls":7,"redis_duration_s":0.001141,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000471,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008155,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.012589,"completed_at":"2026-08-15T13:25:05.466Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.040965] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.467Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"339595ca55cc256919f4b25a","created_at":"2026-08-15T13:25:04.382Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93648daa8df88c3c539eea6de276f890","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.386Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-339595ca55cc256919f4b25a: start","job_status":"start","queue_duration_s":1.081454,"scheduling_latency_s":1.08147} gitlab # [ 278.055478] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.488Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"339595ca55cc256919f4b25a","created_at":"2026-08-15T13:25:04.382Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"93648daa8df88c3c539eea6de276f890","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.386Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-339595ca55cc256919f4b25a: done: 0.020695 sec","job_status":"done","queue_duration_s":1.081454,"scheduling_latency_s":1.08147,"redis_calls":10,"redis_duration_s":0.0014190000000000001,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000593,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012879,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00580282,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.020695,"completed_at":"2026-08-15T13:25:05.488Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00343,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 278.098416] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.489Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"58852b07dfe41be4c3b0e2b7","created_at":"2026-08-15T13:25:04.394Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d3e1f1b580dd83419685b434759bada","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.398Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-58852b07dfe41be4c3b0e2b7: start","job_status":"start","queue_duration_s":1.091556,"scheduling_latency_s":1.091572} gitlab # [ 278.112880] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.510Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"58852b07dfe41be4c3b0e2b7","created_at":"2026-08-15T13:25:04.394Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d3e1f1b580dd83419685b434759bada","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.398Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-58852b07dfe41be4c3b0e2b7: done: 0.020378 sec","job_status":"done","queue_duration_s":1.091556,"scheduling_latency_s":1.091572,"redis_calls":7,"redis_duration_s":0.00096,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000697,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000263,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020378,"completed_at":"2026-08-15T13:25:05.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005605,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.153232] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.511Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9634de124ab73492199afbd8","created_at":"2026-08-15T13:25:04.401Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.403Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9634de124ab73492199afbd8: start","job_status":"start","queue_duration_s":1.107703,"scheduling_latency_s":1.107718} gitlab # [ 278.166836] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.536Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9634de124ab73492199afbd8","created_at":"2026-08-15T13:25:04.401Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-16b80a43a9034512","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:25:04.403Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9634de124ab73492199afbd8: done: 0.025095 sec","job_status":"done","queue_duration_s":1.107703,"scheduling_latency_s":1.107718,"redis_calls":14,"redis_duration_s":0.0018750000000000001,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000211,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001262,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000402,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018875,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025095,"completed_at":"2026-08-15T13:25:05.536Z","load_balancing_strategy":"primary","db_duration_s":0.001841,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.208217] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c5ef6592c7edfac8eb04340","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c5ef6592c7edfac8eb04340: start","job_status":"start","queue_duration_s":0.281393,"scheduling_latency_s":0.281409} gitlab # [ 278.221569] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4c5ef6592c7edfac8eb04340","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4c5ef6592c7edfac8eb04340: done: 0.014688 sec","job_status":"done","queue_duration_s":0.281393,"scheduling_latency_s":0.281409,"redis_calls":13,"redis_duration_s":0.001584,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010135,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014688,"completed_at":"2026-08-15T13:25:05.552Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.260605] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.553Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"078534a422a9c306c6f73e12","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-078534a422a9c306c6f73e12: start","job_status":"start","queue_duration_s":0.297179,"scheduling_latency_s":0.297195} gitlab # [ 278.272995] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"078534a422a9c306c6f73e12","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-078534a422a9c306c6f73e12: done: 0.014552 sec","job_status":"done","queue_duration_s":0.297179,"scheduling_latency_s":0.297195,"redis_calls":13,"redis_duration_s":0.001774,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000802,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000972,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009586,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014552,"completed_at":"2026-08-15T13:25:05.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.313314] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.569Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc7a1a5d0f925ca5cb788fe7","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc7a1a5d0f925ca5cb788fe7: start","job_status":"start","queue_duration_s":0.313174,"scheduling_latency_s":0.31319} gitlab # [ 278.325618] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fc7a1a5d0f925ca5cb788fe7","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fc7a1a5d0f925ca5cb788fe7: done: 0.014222 sec","job_status":"done","queue_duration_s":0.313174,"scheduling_latency_s":0.31319,"redis_calls":13,"redis_duration_s":0.001528,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00097,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009821,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014222,"completed_at":"2026-08-15T13:25:05.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.365170] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.584Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fc6cad0b2ee92022226e0e4","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fc6cad0b2ee92022226e0e4: start","job_status":"start","queue_duration_s":0.328453,"scheduling_latency_s":0.328468} gitlab # [ 278.377381] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.598Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0fc6cad0b2ee92022226e0e4","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0fc6cad0b2ee92022226e0e4: done: 0.014317 sec","job_status":"done","queue_duration_s":0.328453,"scheduling_latency_s":0.328468,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014317,"completed_at":"2026-08-15T13:25:05.598Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.416744] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5937a499f414d6a80c025ddd","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5937a499f414d6a80c025ddd: start","job_status":"start","queue_duration_s":0.343857,"scheduling_latency_s":0.343872} gitlab # [ 278.429809] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.614Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5937a499f414d6a80c025ddd","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5937a499f414d6a80c025ddd: done: 0.014233 sec","job_status":"done","queue_duration_s":0.343857,"scheduling_latency_s":0.343872,"redis_calls":13,"redis_duration_s":0.001448,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000515,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009895,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014233,"completed_at":"2026-08-15T13:25:05.614Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.469052] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.615Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2ae863ef81541f4f1bef816","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2ae863ef81541f4f1bef816: start","job_status":"start","queue_duration_s":0.359168,"scheduling_latency_s":0.359184} gitlab # [ 278.481526] setup-gitlab-start[3003]: W, [2026-08-15T13:25:06.253870 #3003] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 278.483452] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.629Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b2ae863ef81541f4f1bef816","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b2ae863ef81541f4f1bef816: done: 0.013861 sec","job_status":"done","queue_duration_s":0.359168,"scheduling_latency_s":0.359184,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013861,"completed_at":"2026-08-15T13:25:05.629Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.523205] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.630Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930957a5aa8394ad5366fad8","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930957a5aa8394ad5366fad8: start","job_status":"start","queue_duration_s":0.374189,"scheduling_latency_s":0.374205} gitlab # [ 278.535859] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.644Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"930957a5aa8394ad5366fad8","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-930957a5aa8394ad5366fad8: done: 0.013693 sec","job_status":"done","queue_duration_s":0.374189,"scheduling_latency_s":0.374205,"redis_calls":13,"redis_duration_s":0.001496,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00098,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009475,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013693,"completed_at":"2026-08-15T13:25:05.643Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.576118] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.645Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2b7ff0ad91b256941de0ad8","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2b7ff0ad91b256941de0ad8: start","job_status":"start","queue_duration_s":0.388999,"scheduling_latency_s":0.389014} gitlab # [ 278.588676] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.659Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2b7ff0ad91b256941de0ad8","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2b7ff0ad91b256941de0ad8: done: 0.014062 sec","job_status":"done","queue_duration_s":0.388999,"scheduling_latency_s":0.389014,"redis_calls":13,"redis_duration_s":0.0015170000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000964,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014062,"completed_at":"2026-08-15T13:25:05.659Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.629195] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.660Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"164da6f9555eb9007c68ce4b","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-164da6f9555eb9007c68ce4b: start","job_status":"start","queue_duration_s":0.404099,"scheduling_latency_s":0.404114} gitlab # [ 278.641683] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"164da6f9555eb9007c68ce4b","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-164da6f9555eb9007c68ce4b: done: 0.014299 sec","job_status":"done","queue_duration_s":0.404099,"scheduling_latency_s":0.404114,"redis_calls":13,"redis_duration_s":0.001698,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001144,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009862,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014299,"completed_at":"2026-08-15T13:25:05.674Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.680720] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.675Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f544fb2897e99db77a18123","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f544fb2897e99db77a18123: start","job_status":"start","queue_duration_s":0.419495,"scheduling_latency_s":0.419511} gitlab # [ 278.693164] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8f544fb2897e99db77a18123","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8f544fb2897e99db77a18123: done: 0.013602 sec","job_status":"done","queue_duration_s":0.419495,"scheduling_latency_s":0.419511,"redis_calls":13,"redis_duration_s":0.001437,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009481,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013602,"completed_at":"2026-08-15T13:25:05.689Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.732426] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4aa1419a81313fa214a9d0","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4aa1419a81313fa214a9d0: start","job_status":"start","queue_duration_s":0.434152,"scheduling_latency_s":0.434168} gitlab # [ 278.744681] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9d4aa1419a81313fa214a9d0","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9d4aa1419a81313fa214a9d0: done: 0.013904 sec","job_status":"done","queue_duration_s":0.434152,"scheduling_latency_s":0.434168,"redis_calls":13,"redis_duration_s":0.0015300000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009707,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013904,"completed_at":"2026-08-15T13:25:05.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.784044] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.705Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5be9f8a3e61a465297432301","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5be9f8a3e61a465297432301: start","job_status":"start","queue_duration_s":0.44912,"scheduling_latency_s":0.449136} gitlab # [ 278.796226] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.718Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5be9f8a3e61a465297432301","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5be9f8a3e61a465297432301: done: 0.013711 sec","job_status":"done","queue_duration_s":0.44912,"scheduling_latency_s":0.449136,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009559,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013711,"completed_at":"2026-08-15T13:25:05.718Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.836339] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.720Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07de66bb7ea4146f21e0ab70","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07de66bb7ea4146f21e0ab70: start","job_status":"start","queue_duration_s":0.463867,"scheduling_latency_s":0.463882} gitlab # [ 278.848579] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.733Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"07de66bb7ea4146f21e0ab70","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-07de66bb7ea4146f21e0ab70: done: 0.01373 sec","job_status":"done","queue_duration_s":0.463867,"scheduling_latency_s":0.463882,"redis_calls":13,"redis_duration_s":0.001436,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009407,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01373,"completed_at":"2026-08-15T13:25:05.733Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.887931] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.734Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56c9ff69e8f0c4436d4b5b71","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56c9ff69e8f0c4436d4b5b71: start","job_status":"start","queue_duration_s":0.478649,"scheduling_latency_s":0.478665} gitlab # [ 278.900367] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.749Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"56c9ff69e8f0c4436d4b5b71","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-56c9ff69e8f0c4436d4b5b71: done: 0.014938 sec","job_status":"done","queue_duration_s":0.478649,"scheduling_latency_s":0.478665,"redis_calls":13,"redis_duration_s":0.002243,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001191,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001052,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009662,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014938,"completed_at":"2026-08-15T13:25:05.749Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.939524] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.750Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bbf93289a1354de1c650b23","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bbf93289a1354de1c650b23: start","job_status":"start","queue_duration_s":0.494665,"scheduling_latency_s":0.494704} gitlab # [ 278.952122] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8bbf93289a1354de1c650b23","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8bbf93289a1354de1c650b23: done: 0.018466 sec","job_status":"done","queue_duration_s":0.494665,"scheduling_latency_s":0.494704,"redis_calls":13,"redis_duration_s":0.0019359999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001027,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018466,"completed_at":"2026-08-15T13:25:05.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 278.991353] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0936d2c3538b48431f2f6cc3","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0936d2c3538b48431f2f6cc3: start","job_status":"start","queue_duration_s":0.514196,"scheduling_latency_s":0.514212} gitlab # [ 279.003979] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.784Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0936d2c3538b48431f2f6cc3","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0936d2c3538b48431f2f6cc3: done: 0.013682 sec","job_status":"done","queue_duration_s":0.514196,"scheduling_latency_s":0.514212,"redis_calls":13,"redis_duration_s":0.0014529999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000904,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013682,"completed_at":"2026-08-15T13:25:05.783Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.045137] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6a9dda71ff924013347f8a9","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6a9dda71ff924013347f8a9: start","job_status":"start","queue_duration_s":0.528934,"scheduling_latency_s":0.528949} gitlab # [ 279.057305] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.798Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d6a9dda71ff924013347f8a9","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d6a9dda71ff924013347f8a9: done: 0.013778 sec","job_status":"done","queue_duration_s":0.528934,"scheduling_latency_s":0.528949,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000596,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013778,"completed_at":"2026-08-15T13:25:05.798Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.096287] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.800Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"520a126be11422ba2f61c523","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-520a126be11422ba2f61c523: start","job_status":"start","queue_duration_s":0.543936,"scheduling_latency_s":0.543952} gitlab # [ 279.108615] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.814Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"520a126be11422ba2f61c523","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-520a126be11422ba2f61c523: done: 0.013991 sec","job_status":"done","queue_duration_s":0.543936,"scheduling_latency_s":0.543952,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000939,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009736,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013991,"completed_at":"2026-08-15T13:25:05.814Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.147761] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.815Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d07ec8a0ccbf795ebd5ba473","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d07ec8a0ccbf795ebd5ba473: start","job_status":"start","queue_duration_s":0.559011,"scheduling_latency_s":0.559026} gitlab # [ 279.160444] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d07ec8a0ccbf795ebd5ba473","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d07ec8a0ccbf795ebd5ba473: done: 0.013526 sec","job_status":"done","queue_duration_s":0.559011,"scheduling_latency_s":0.559026,"redis_calls":13,"redis_duration_s":0.001409,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000506,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000903,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009388,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013526,"completed_at":"2026-08-15T13:25:05.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.199362] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2056e20db9636792a30500e1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2056e20db9636792a30500e1: start","job_status":"start","queue_duration_s":0.573584,"scheduling_latency_s":0.573601} gitlab # [ 279.211967] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.843Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2056e20db9636792a30500e1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2056e20db9636792a30500e1: done: 0.013544 sec","job_status":"done","queue_duration_s":0.573584,"scheduling_latency_s":0.573601,"redis_calls":13,"redis_duration_s":0.001401,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013544,"completed_at":"2026-08-15T13:25:05.843Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.252131] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc8651ed05cedde9fbf095c4","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc8651ed05cedde9fbf095c4: start","job_status":"start","queue_duration_s":0.588262,"scheduling_latency_s":0.588277} gitlab # [ 279.264121] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.858Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bc8651ed05cedde9fbf095c4","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bc8651ed05cedde9fbf095c4: done: 0.013739 sec","job_status":"done","queue_duration_s":0.588262,"scheduling_latency_s":0.588277,"redis_calls":13,"redis_duration_s":0.001444,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013739,"completed_at":"2026-08-15T13:25:05.858Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.303177] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.859Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88247b8de1e4c3afc6d37a79","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88247b8de1e4c3afc6d37a79: start","job_status":"start","queue_duration_s":0.603415,"scheduling_latency_s":0.603431} gitlab # [ 279.315834] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.874Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"88247b8de1e4c3afc6d37a79","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-88247b8de1e4c3afc6d37a79: done: 0.014986 sec","job_status":"done","queue_duration_s":0.603415,"scheduling_latency_s":0.603431,"redis_calls":13,"redis_duration_s":0.001495,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010262,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014986,"completed_at":"2026-08-15T13:25:05.874Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.355045] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.875Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fc13da3831940dba8a85abd","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fc13da3831940dba8a85abd: start","job_status":"start","queue_duration_s":0.619709,"scheduling_latency_s":0.619725} gitlab # [ 279.367567] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.890Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fc13da3831940dba8a85abd","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fc13da3831940dba8a85abd: done: 0.014271 sec","job_status":"done","queue_duration_s":0.619709,"scheduling_latency_s":0.619725,"redis_calls":13,"redis_duration_s":0.001426,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000512,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000914,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014271,"completed_at":"2026-08-15T13:25:05.890Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.406626] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.891Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2891b6913832b0b539a8090","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2891b6913832b0b539a8090: start","job_status":"start","queue_duration_s":0.63546,"scheduling_latency_s":0.635477} gitlab # [ 279.419164] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.906Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2891b6913832b0b539a8090","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2891b6913832b0b539a8090: done: 0.015363 sec","job_status":"done","queue_duration_s":0.63546,"scheduling_latency_s":0.635477,"redis_calls":13,"redis_duration_s":0.001506,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011121,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015363,"completed_at":"2026-08-15T13:25:05.906Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.459074] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.908Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e791460736897fc69846e1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e791460736897fc69846e1: start","job_status":"start","queue_duration_s":0.652337,"scheduling_latency_s":0.652353} gitlab # [ 279.471622] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.923Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"34e791460736897fc69846e1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-34e791460736897fc69846e1: done: 0.015174 sec","job_status":"done","queue_duration_s":0.652337,"scheduling_latency_s":0.652353,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000534,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010903,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015174,"completed_at":"2026-08-15T13:25:05.923Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.510790] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.925Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4f25943ef77e0f79afa5641","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4f25943ef77e0f79afa5641: start","job_status":"start","queue_duration_s":0.669006,"scheduling_latency_s":0.669022} gitlab # [ 279.523759] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.940Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4f25943ef77e0f79afa5641","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4f25943ef77e0f79afa5641: done: 0.014976 sec","job_status":"done","queue_duration_s":0.669006,"scheduling_latency_s":0.669022,"redis_calls":13,"redis_duration_s":0.0014629999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000532,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000931,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014976,"completed_at":"2026-08-15T13:25:05.940Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.563152] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.941Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"341a900bcf7cbacbd279b366","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-341a900bcf7cbacbd279b366: start","job_status":"start","queue_duration_s":0.685031,"scheduling_latency_s":0.685047} gitlab # [ 279.575741] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.956Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"341a900bcf7cbacbd279b366","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-341a900bcf7cbacbd279b366: done: 0.015653 sec","job_status":"done","queue_duration_s":0.685031,"scheduling_latency_s":0.685047,"redis_calls":13,"redis_duration_s":0.001852,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000518,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001334,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015653,"completed_at":"2026-08-15T13:25:05.956Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.615204] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.958Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba7b39d1f7919730369cb94a","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba7b39d1f7919730369cb94a: start","job_status":"start","queue_duration_s":0.701846,"scheduling_latency_s":0.701862} gitlab # [ 279.627836] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.974Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ba7b39d1f7919730369cb94a","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ba7b39d1f7919730369cb94a: done: 0.016043 sec","job_status":"done","queue_duration_s":0.701846,"scheduling_latency_s":0.701862,"redis_calls":13,"redis_duration_s":0.002156,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000524,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001632,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011077,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016043,"completed_at":"2026-08-15T13:25:05.973Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.668446] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.975Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f68141f5f76f97cac499e7cb","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f68141f5f76f97cac499e7cb: start","job_status":"start","queue_duration_s":0.718951,"scheduling_latency_s":0.718966} gitlab # [ 279.680868] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.990Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f68141f5f76f97cac499e7cb","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f68141f5f76f97cac499e7cb: done: 0.015614 sec","job_status":"done","queue_duration_s":0.718951,"scheduling_latency_s":0.718966,"redis_calls":13,"redis_duration_s":0.0023309999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001436,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010401,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015614,"completed_at":"2026-08-15T13:25:05.990Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.720225] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:05.991Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6892a288b317d3555919e0cf","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6892a288b317d3555919e0cf: start","job_status":"start","queue_duration_s":0.735676,"scheduling_latency_s":0.735692} gitlab # [ 279.732607] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.013Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6892a288b317d3555919e0cf","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6892a288b317d3555919e0cf: done: 0.021223 sec","job_status":"done","queue_duration_s":0.735676,"scheduling_latency_s":0.735692,"redis_calls":13,"redis_duration_s":0.001473,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000934,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010678,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021223,"completed_at":"2026-08-15T13:25:06.013Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.771814] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.014Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b4d8415c399644c1969c08a","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b4d8415c399644c1969c08a: start","job_status":"start","queue_duration_s":0.758036,"scheduling_latency_s":0.758053} gitlab # [ 279.784472] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.029Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1b4d8415c399644c1969c08a","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1b4d8415c399644c1969c08a: done: 0.015732 sec","job_status":"done","queue_duration_s":0.758036,"scheduling_latency_s":0.758053,"redis_calls":13,"redis_duration_s":0.001598,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000592,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01082,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015732,"completed_at":"2026-08-15T13:25:06.029Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.823454] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.031Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"965a66356b6988f658976467","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-965a66356b6988f658976467: start","job_status":"start","queue_duration_s":0.775443,"scheduling_latency_s":0.77546} gitlab # [ 279.835901] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.047Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"965a66356b6988f658976467","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-965a66356b6988f658976467: done: 0.015447 sec","job_status":"done","queue_duration_s":0.775443,"scheduling_latency_s":0.77546,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000944,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010611,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015447,"completed_at":"2026-08-15T13:25:06.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.876361] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"692c0256a37284b080713f91","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-692c0256a37284b080713f91: start","job_status":"start","queue_duration_s":0.792443,"scheduling_latency_s":0.79246} gitlab # [ 279.888592] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.063Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"692c0256a37284b080713f91","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-692c0256a37284b080713f91: done: 0.015379 sec","job_status":"done","queue_duration_s":0.792443,"scheduling_latency_s":0.79246,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000542,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011104,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015379,"completed_at":"2026-08-15T13:25:06.063Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.928451] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.065Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b550aca54546556f74a3955","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b550aca54546556f74a3955: start","job_status":"start","queue_duration_s":0.809453,"scheduling_latency_s":0.80947} gitlab # [ 279.940765] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.081Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5b550aca54546556f74a3955","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5b550aca54546556f74a3955: done: 0.015897 sec","job_status":"done","queue_duration_s":0.809453,"scheduling_latency_s":0.80947,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000519,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015897,"completed_at":"2026-08-15T13:25:06.081Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 279.980243] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.083Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d16954abcf37b660306dde95","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d16954abcf37b660306dde95: start","job_status":"start","queue_duration_s":0.826972,"scheduling_latency_s":0.826988} gitlab # [ 279.992543] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.096Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d16954abcf37b660306dde95","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d16954abcf37b660306dde95: done: 0.013856 sec","job_status":"done","queue_duration_s":0.826972,"scheduling_latency_s":0.826988,"redis_calls":13,"redis_duration_s":0.0015379999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000998,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013856,"completed_at":"2026-08-15T13:25:06.096Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.031857] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.098Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7346e6fa09eb2e9ddc42b8ad","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7346e6fa09eb2e9ddc42b8ad: start","job_status":"start","queue_duration_s":0.841914,"scheduling_latency_s":0.841929} gitlab # [ 280.044519] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.111Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7346e6fa09eb2e9ddc42b8ad","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7346e6fa09eb2e9ddc42b8ad: done: 0.013448 sec","job_status":"done","queue_duration_s":0.841914,"scheduling_latency_s":0.841929,"redis_calls":13,"redis_duration_s":0.001395,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000504,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000891,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009316,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013448,"completed_at":"2026-08-15T13:25:06.111Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.084177] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dead69cd1cb61884a19eda1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dead69cd1cb61884a19eda1: start","job_status":"start","queue_duration_s":0.856394,"scheduling_latency_s":0.856409} gitlab # [ 280.096310] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.126Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7dead69cd1cb61884a19eda1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7dead69cd1cb61884a19eda1: done: 0.013564 sec","job_status":"done","queue_duration_s":0.856394,"scheduling_latency_s":0.856409,"redis_calls":13,"redis_duration_s":0.001413,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000516,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013564,"completed_at":"2026-08-15T13:25:06.126Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.135775] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.127Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f46e8515b46d5c0d38be95e5","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f46e8515b46d5c0d38be95e5: start","job_status":"start","queue_duration_s":0.871066,"scheduling_latency_s":0.871081} gitlab # [ 280.148205] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.141Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f46e8515b46d5c0d38be95e5","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f46e8515b46d5c0d38be95e5: done: 0.014232 sec","job_status":"done","queue_duration_s":0.871066,"scheduling_latency_s":0.871081,"redis_calls":13,"redis_duration_s":0.001743,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001215,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009608,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014232,"completed_at":"2026-08-15T13:25:06.141Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.187082] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.142Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500c7bf0a22fcfe126f5dfb7","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500c7bf0a22fcfe126f5dfb7: start","job_status":"start","queue_duration_s":0.886389,"scheduling_latency_s":0.886405} gitlab # [ 280.199622] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.156Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"500c7bf0a22fcfe126f5dfb7","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-500c7bf0a22fcfe126f5dfb7: done: 0.013592 sec","job_status":"done","queue_duration_s":0.886389,"scheduling_latency_s":0.886405,"redis_calls":13,"redis_duration_s":0.001414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000513,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000901,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009439,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013592,"completed_at":"2026-08-15T13:25:06.156Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.238705] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.157Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75d5ce7e4f97c23d5934aa0","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75d5ce7e4f97c23d5934aa0: start","job_status":"start","queue_duration_s":0.901055,"scheduling_latency_s":0.901071} gitlab # [ 280.251221] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.171Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f75d5ce7e4f97c23d5934aa0","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f75d5ce7e4f97c23d5934aa0: done: 0.013913 sec","job_status":"done","queue_duration_s":0.901055,"scheduling_latency_s":0.901071,"redis_calls":13,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001133,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009453,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013913,"completed_at":"2026-08-15T13:25:06.171Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.291424] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.172Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08426e98bc5d24f9bfde0519","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08426e98bc5d24f9bfde0519: start","job_status":"start","queue_duration_s":0.916042,"scheduling_latency_s":0.916058} gitlab # [ 280.303988] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08426e98bc5d24f9bfde0519","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08426e98bc5d24f9bfde0519: done: 0.013605 sec","job_status":"done","queue_duration_s":0.916042,"scheduling_latency_s":0.916058,"redis_calls":13,"redis_duration_s":0.001425,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000528,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000897,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009414,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013605,"completed_at":"2026-08-15T13:25:06.185Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.343092] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.186Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c467eec574a39226b094062","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c467eec574a39226b094062: start","job_status":"start","queue_duration_s":0.930735,"scheduling_latency_s":0.930751} gitlab # [ 280.355391] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9c467eec574a39226b094062","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9c467eec574a39226b094062: done: 0.013529 sec","job_status":"done","queue_duration_s":0.930735,"scheduling_latency_s":0.930751,"redis_calls":13,"redis_duration_s":0.001423,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000507,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000916,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009391,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013529,"completed_at":"2026-08-15T13:25:06.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.394522] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08390759200de2b4a3c608db","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08390759200de2b4a3c608db: start","job_status":"start","queue_duration_s":0.945301,"scheduling_latency_s":0.945316} gitlab # [ 280.406957] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08390759200de2b4a3c608db","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08390759200de2b4a3c608db: done: 0.013869 sec","job_status":"done","queue_duration_s":0.945301,"scheduling_latency_s":0.945316,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009574,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013869,"completed_at":"2026-08-15T13:25:06.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.445956] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.216Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff09118570937df8d4692c18","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff09118570937df8d4692c18: start","job_status":"start","queue_duration_s":0.960273,"scheduling_latency_s":0.96029} gitlab # [ 280.458469] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.230Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff09118570937df8d4692c18","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff09118570937df8d4692c18: done: 0.013614 sec","job_status":"done","queue_duration_s":0.960273,"scheduling_latency_s":0.96029,"redis_calls":13,"redis_duration_s":0.001412,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00051,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000902,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009385,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013614,"completed_at":"2026-08-15T13:25:06.229Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.499010] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def602f0f41f15a568cdb821","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def602f0f41f15a568cdb821: start","job_status":"start","queue_duration_s":0.974999,"scheduling_latency_s":0.975015} gitlab # [ 280.511684] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.245Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"def602f0f41f15a568cdb821","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-def602f0f41f15a568cdb821: done: 0.014248 sec","job_status":"done","queue_duration_s":0.974999,"scheduling_latency_s":0.975015,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009839,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014248,"completed_at":"2026-08-15T13:25:06.245Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.550616] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.246Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e75206dab7a32390d570c152","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e75206dab7a32390d570c152: start","job_status":"start","queue_duration_s":0.990375,"scheduling_latency_s":0.990391} gitlab # [ 280.562984] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.260Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e75206dab7a32390d570c152","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e75206dab7a32390d570c152: done: 0.013648 sec","job_status":"done","queue_duration_s":0.990375,"scheduling_latency_s":0.990391,"redis_calls":13,"redis_duration_s":0.0014269999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000508,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000919,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013648,"completed_at":"2026-08-15T13:25:06.260Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.602094] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.261Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967978107bd18f859cdb914f","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967978107bd18f859cdb914f: start","job_status":"start","queue_duration_s":1.005155,"scheduling_latency_s":1.005172} gitlab # [ 280.614301] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"967978107bd18f859cdb914f","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-967978107bd18f859cdb914f: done: 0.014099 sec","job_status":"done","queue_duration_s":1.005155,"scheduling_latency_s":1.005172,"redis_calls":13,"redis_duration_s":0.001725,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000559,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001166,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014099,"completed_at":"2026-08-15T13:25:06.275Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.653147] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.276Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08fe38ed322992c63c2707fb","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08fe38ed322992c63c2707fb: start","job_status":"start","queue_duration_s":1.020321,"scheduling_latency_s":1.020338} gitlab # [ 280.666700] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.290Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"08fe38ed322992c63c2707fb","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-08fe38ed322992c63c2707fb: done: 0.0136 sec","job_status":"done","queue_duration_s":1.020321,"scheduling_latency_s":1.020338,"redis_calls":13,"redis_duration_s":0.001439,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000521,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000918,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0136,"completed_at":"2026-08-15T13:25:06.290Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.705557] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.291Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7993ffc01e86e289d4f344f1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7993ffc01e86e289d4f344f1: start","job_status":"start","queue_duration_s":1.035053,"scheduling_latency_s":1.035069} gitlab # [ 280.718223] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.304Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7993ffc01e86e289d4f344f1","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7993ffc01e86e289d4f344f1: done: 0.013589 sec","job_status":"done","queue_duration_s":1.035053,"scheduling_latency_s":1.035069,"redis_calls":13,"redis_duration_s":0.0014030000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000496,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000907,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009417,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013589,"completed_at":"2026-08-15T13:25:06.304Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.757071] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.305Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e402c0d572cd19eb0fc425","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e402c0d572cd19eb0fc425: start","job_status":"start","queue_duration_s":1.049763,"scheduling_latency_s":1.049779} gitlab # [ 280.769686] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.319Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b4e402c0d572cd19eb0fc425","created_at":"2026-08-15T13:25:05.181Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-2cf0b351986148c9","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"fbca19a2f7d39556a0cbfd18c4015c05","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.256Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b4e402c0d572cd19eb0fc425: done: 0.013928 sec","job_status":"done","queue_duration_s":1.049763,"scheduling_latency_s":1.049779,"redis_calls":13,"redis_duration_s":0.0015530000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000989,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009619,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.013928,"completed_at":"2026-08-15T13:25:06.319Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.808808] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.320Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae1226eb7524f6ed1708cca2","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae1226eb7524f6ed1708cca2: start","job_status":"start","queue_duration_s":0.789893,"scheduling_latency_s":0.789909} gitlab # [ 280.820969] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.332Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.822552] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.340Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"ae1226eb7524f6ed1708cca2","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-ae1226eb7524f6ed1708cca2: done: 0.02003 sec","job_status":"done","queue_duration_s":0.789893,"scheduling_latency_s":0.789909,"redis_calls":10,"redis_duration_s":0.001299,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02003,"completed_at":"2026-08-15T13:25:06.340Z","load_balancing_strategy":"primary","db_duration_s":0.003534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.860618] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.342Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f2cbe46b2a5423021ecc855","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f2cbe46b2a5423021ecc855: start","job_status":"start","queue_duration_s":0.811054,"scheduling_latency_s":0.81107} gitlab # [ 280.873909] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.352Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.875579] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.360Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"8f2cbe46b2a5423021ecc855","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-8f2cbe46b2a5423021ecc855: done: 0.018476 sec","job_status":"done","queue_duration_s":0.811054,"scheduling_latency_s":0.81107,"redis_calls":10,"redis_duration_s":0.001305,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000748,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012571,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018476,"completed_at":"2026-08-15T13:25:06.360Z","load_balancing_strategy":"primary","db_duration_s":0.002688,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.914068] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.361Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ae24f82ceb50d79b8a821d9","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ae24f82ceb50d79b8a821d9: start","job_status":"start","queue_duration_s":0.830698,"scheduling_latency_s":0.830714} gitlab # [ 280.926467] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.372Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.928147] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.380Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9ae24f82ceb50d79b8a821d9","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9ae24f82ceb50d79b8a821d9: done: 0.01872 sec","job_status":"done","queue_duration_s":0.830698,"scheduling_latency_s":0.830714,"redis_calls":10,"redis_duration_s":0.001382,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000576,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000806,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012745,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01872,"completed_at":"2026-08-15T13:25:06.380Z","load_balancing_strategy":"primary","db_duration_s":0.002484,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 280.966361] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.381Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d05f8d4f3291cbc0bdbea5dd","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d05f8d4f3291cbc0bdbea5dd: start","job_status":"start","queue_duration_s":0.85051,"scheduling_latency_s":0.850526} gitlab # [ 280.978764] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.392Z","message":"No existing merge request to be cleaned up."} gitlab # [ 280.980424] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:06.400Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d05f8d4f3291cbc0bdbea5dd","created_at":"2026-08-15T13:25:05.522Z","trace_propagation_headers":{"sentry-trace":"e739b108b04c481daa9da49a5922a6a7-476253c1caed4da2","baggage":"sentry-trace_id=e739b108b04c481daa9da49a5922a6a7,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"8622c4169f9a615bef47075b547eaca1","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:25:05.531Z","job_size_bytes":2,"pid":2046,"sidekiq_tid":"maom","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d05f8d4f3291cbc0bdbea5dd: done: 0.018609 sec","job_status":"done","queue_duration_s":0.85051,"scheduling_latency_s":0.850526,"redis_calls":10,"redis_duration_s":0.001297,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000735,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012541,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018609,"completed_at":"2026-08-15T13:25:06.400Z","load_balancing_strategy":"primary","db_duration_s":0.002802,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 310.243773] redis[785]: 10 changes in 300 seconds. Saving... gitlab # [ 310.250959] redis[785]: Background saving started by pid 3319 gitlab # [ 310.255654] redis[3319]: BGSAVE done, 473 keys saved, 0 keys skipped, 121371 bytes written. gitlab # [ 310.260184] redis[3319]: DB saved on disk gitlab # [ 310.262360] redis[3319]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB gitlab # [ 310.344947] redis[785]: Background saving terminated with success gitlab # [ 311.418913] postgres[862]: [862] LOG: checkpoint starting: time gitlab # [ 312.969281] gitaly[904]: time="2026-08-15T13:25:40.798Z" level=info msg="burden monitor stats" pid=904 gitlab # [ 320.186070] postgres[3337]: [3337] ERROR: relation "table_without_models" does not exist gitlab # [ 320.186445] postgres[3337]: [3337] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab # [ 325.846478] runuser[3002]: pam_unix(runuser:session): session closed for user gitlab gitlab # [ 325.847943] setup-gitlab-start[2976]: + systemctl restart gitlab-sidekiq.service gitlab # [ 325.887894] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:53.717Z","message":"Shutting down"} gitlab # [ 325.892477] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:53.717Z","message":"Scheduler exiting..."} gitlab # [ 325.896811] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:53.719Z","message":"Terminating quiet threads for default capsule"} gitlab # [ 325.901928] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:53.719Z","message":"Scheduler exiting..."} gitlab # [ 325.907005] systemd[1]: Stopping gitlab-sidekiq.service... gitlab # [ 326.390007] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:54.220Z","message":"Pausing to allow jobs to finish..."} gitlab # [ 328.896315] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:56.725Z","message":"Bye!"} gitlab # [ 329.422312] sidekiq-cluster[2046]: {"severity":"INFO","time":"2026-08-15T13:25:57.252Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":2046,"worker_id":"sidekiq_0","memwd_rss_bytes":920829952,"retry":0} gitlab # [ 330.339721] systemd[1]: gitlab-sidekiq.service: Deactivated successfully. gitlab # [ 330.342952] systemd[1]: Stopped gitlab-sidekiq.service. gitlab # [ 330.345394] systemd[1]: gitlab-sidekiq.service: Consumed 1min 51.064s CPU time over 3min 23.697s wall clock time, 936.9M memory peak, 4.6M read from disk. gitlab # [ 330.351496] systemd[1]: Started gitlab-sidekiq.service. gitlab # [ 330.354825] setup-gitlab-start[2976]: + mkdir -p /var/lib/secrets gitlab # [ 330.358372] setup-gitlab-start[2976]: + echo nixbot-test-token-of-20-chars gitlab # [ 330.361387] setup-gitlab-start[2976]: + chmod 644 /var/lib/secrets/gitlab-token gitlab # [ 330.364556] setup-gitlab-start[2976]: + curl -fs -X POST -H 'PRIVATE-TOKEN: nixbot-test-token-of-20-chars' -H 'Content-Type: application/json' -d '{"name":"test-flake","visibility":"public","topics":["build-with-buildbot"]}' http://gitlab/api/v4/projects gitlab # [ 332.075927] gitaly[904]: time="2026-08-15T13:25:59.905Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:09.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:25:59.904" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.106024] gitaly[904]: time="2026-08-15T13:25:59.905Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:09.002" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:25:59.904" grpc.time_ms=0.731 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.371255] gitaly[904]: time="2026-08-15T13:26:00.201Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.201" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.393719] gitaly[904]: time="2026-08-15T13:26:00.202Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.010" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.201" grpc.time_ms=0.582 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.464519] gitaly[904]: time="2026-08-15T13:26:00.295Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 diskcache=61a51343-49f3-49e5-b1e9-d70e76411b0b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-15T13:26:10.027" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.218" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.492869] gitaly[904]: time="2026-08-15T13:26:00.295Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=36 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1444 command.oublock=32 command.real_time_ms=50 command.spawn_token_fork_ms=5 command.system_time_ms=28 command.user_time_ms=8 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-15T13:26:10.027" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.218" grpc.time_ms=76.822 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.517476] gitaly[904]: time="2026-08-15T13:26:00.320Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.128" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.319" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 332.535423] gitaly[904]: time="2026-08-15T13:26:00.320Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.128" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.319" grpc.time_ms=1.407 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.552631] gitaly[904]: time="2026-08-15T13:26:00.346Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=6 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:26:10.138" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.329" grpc.time_ms=17.154 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.643872] gitaly[904]: time="2026-08-15T13:26:00.474Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.283" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.473" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.676161] gitaly[904]: time="2026-08-15T13:26:00.474Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.283" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.473" grpc.time_ms=0.589 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.708560] gitaly[904]: time="2026-08-15T13:26:00.524Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 diskcache=ad121274-cc08-4f51-92db-7b2ab63467b3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-15T13:26:10.287" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.478" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.732272] gitaly[904]: time="2026-08-15T13:26:00.524Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1441 command.oublock=32 command.real_time_ms=23 command.spawn_token_fork_ms=5 command.system_time_ms=9 command.user_time_ms=10 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-08-15T13:26:10.287" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.478" grpc.time_ms=45.804 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.757640] gitaly[904]: time="2026-08-15T13:26:00.536Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.535" pid=904 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root gitlab # [ 332.773369] gitaly[904]: time="2026-08-15T13:26:00.536Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.535" grpc.time_ms=1.327 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.788837] gitaly[904]: time="2026-08-15T13:26:00.559Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:26:10.014" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.544" grpc.time_ms=14.974 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.807091] gitaly[904]: time="2026-08-15T13:26:00.621Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:26:10.090" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.621" grpc.time_ms=0.368 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 332.820163] gitaly[904]: time="2026-08-15T13:26:00.635Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-15T13:26:10.103" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:00.634" grpc.time_ms=0.756 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 333.447262] gitaly[904]: time="2026-08-15T13:26:01.276Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKGTEATY7ZXMX2ECEKRA9 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:26:11.070" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:01.263" grpc.time_ms=13.446 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 333.897797] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:01 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 3504 gitlab # [ 333.905414] setup-gitlab-start[2976]: + mkdir -p /tmp/test-flake gitlab # [ 333.908536] setup-gitlab-start[2976]: + cd /tmp/test-flake gitlab # [ 333.911803] setup-gitlab-start[2976]: + git init -b master gitlab # [ 333.950677] setup-gitlab-start[3455]: Initialized empty Git repository in /tmp/test-flake/.git/ gitlab # [ 333.954426] setup-gitlab-start[2976]: + git config user.name test gitlab # [ 333.957056] setup-gitlab-start[2976]: + git config user.email test@example.com gitlab # [ 333.960772] setup-gitlab-start[2976]: + cat gitlab # [ 333.964962] setup-gitlab-start[2976]: + git add flake.nix gitlab # [ 333.971389] setup-gitlab-start[2976]: + git commit -m 'initial commit' gitlab # [ 334.006596] setup-gitlab-start[3460]: [master (root-commit) 4b2631e] initial commit gitlab # [ 334.009591] setup-gitlab-start[3460]: 1 file changed, 10 insertions(+) gitlab # [ 334.011994] setup-gitlab-start[3460]: create mode 100644 flake.nix gitlab # [ 334.014343] setup-gitlab-start[2976]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git gitlab # [ 334.020261] setup-gitlab-start[2976]: + git push -u origin master gitlab # [ 334.154805] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:01 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 83 gitlab # [ 334.326417] gitaly[904]: time="2026-08-15T13:26:02.157Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMG2FZ4VJ4V9CG2VZ8M1 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:02.137" grpc.time_ms=19.164 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 334.348399] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:02 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 191 gitlab # [ 334.724886] gitaly[904]: time="2026-08-15T13:26:02.553Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:26:12.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:02.536" grpc.time_ms=16.206 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.792854] gitaly[904]: time="2026-08-15T13:26:02.623Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-15T13:26:32.009" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-15T13:26:02.609" grpc.time_ms=13.705 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 334.831932] gitaly[904]: time="2026-08-15T13:26:02.657Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 duration_ms=251ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 334.870157] gitaly[904]: time="2026-08-15T13:26:02.700Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 duration_ms=41ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 334.879787] gitaly[904]: time="2026-08-15T13:26:02.701Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:02.403" grpc.time_ms=297.388 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 334.937720] gitaly[904]: time="2026-08-15T13:26:02.768Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:02.766" grpc.time_ms=1.303 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.005672] gitaly[904]: time="2026-08-15T13:26:02.836Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:02.834" grpc.time_ms=1.087 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.047572] gitaly[904]: time="2026-08-15T13:26:02.878Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:02.876" grpc.time_ms=1.598 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.265988] sidekiq-cluster[3403]: {"severity":"INFO","time":"2026-08-15T13:26:03.066Z","message":"Starting cluster with 1 processes"} gitlab # [ 335.499356] gitaly[904]: time="2026-08-15T13:26:03.329Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:26:13.006" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:03.316" grpc.time_ms=12.906 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.624470] gitaly[904]: time="2026-08-15T13:26:03.454Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=31 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1433 command.oublock=0 command.real_time_ms=35 command.spawn_token_fork_ms=2 command.system_time_ms=31 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-15T13:26:13.105" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:03.415" grpc.time_ms=38.927 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 335.707378] gitaly[904]: time="2026-08-15T13:26:03.537Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 duration_ms=611ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 335.721000] gitaly[904]: time="2026-08-15T13:26:03.539Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:02.924" grpc.time_ms=615.264 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.753531] gitaly[904]: time="2026-08-15T13:26:03.557Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 diskcache=66ffe2f1-7808-45bb-aec0-cba595fa247b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:02.298" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.776095] systemd[1]: Finished setup-gitlab.service. gitlab # [ 335.777917] gitlab-workhorse[820]: time="2026-08-15T13:26:03Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # [ 335.781183] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:03 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 1359 gitlab # [ 335.784705] setup-gitlab-start[3464]: To http://gitlab/root/test-flake.git gitlab # [ 335.786827] setup-gitlab-start[3464]: * [new branch] master -> master gitlab # [ 335.788388] setup-gitlab-start[3464]: branch 'master' set up to track 'origin/master'. gitlab # [ 335.790112] gitaly[904]: time="2026-08-15T13:26:03.557Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=315 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13421 command.oublock=32 command.real_time_ms=1250 command.spawn_token_fork_ms=2 command.system_time_ms=185 command.user_time_ms=130 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:02.298" grpc.time_ms=1258.982 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 335.812369] systemd[1]: Starting nixbot CI service... gitlab: (finished: waiting for unit setup-gitlab.service, in 336.81 seconds) gitlab: waiting for unit nixbot.service gitlab # [ 337.262183] nixbot[3556]: {"ts": "2026-08-15T13:26:05.091Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"} gitlab # [ 337.288293] nixbot[3556]: {"ts": "2026-08-15T13:26:05.118Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"} gitlab # [ 337.306582] nixbot[3556]: {"ts": "2026-08-15T13:26:05.137Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"} gitlab # [ 337.313259] nixbot[3556]: {"ts": "2026-08-15T13:26:05.141Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"} gitlab # [ 337.319918] nixbot[3556]: {"ts": "2026-08-15T13:26:05.146Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"} gitlab # [ 337.326518] nixbot[3556]: {"ts": "2026-08-15T13:26:05.152Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"} gitlab # [ 337.332943] nixbot[3556]: {"ts": "2026-08-15T13:26:05.155Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"} gitlab # [ 337.339588] nixbot[3556]: {"ts": "2026-08-15T13:26:05.157Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"} gitlab # [ 337.346235] nixbot[3556]: {"ts": "2026-08-15T13:26:05.162Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"} gitlab # [ 337.353036] nixbot[3556]: {"ts": "2026-08-15T13:26:05.164Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"} gitlab # [ 337.359862] nixbot[3556]: {"ts": "2026-08-15T13:26:05.169Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"} gitlab # [ 337.366548] nixbot[3556]: {"ts": "2026-08-15T13:26:05.174Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"} gitlab # [ 337.373327] nixbot[3556]: {"ts": "2026-08-15T13:26:05.176Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"} gitlab # [ 337.379758] nixbot[3556]: {"ts": "2026-08-15T13:26:05.178Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"} gitlab # [ 337.385614] nixbot[3556]: {"ts": "2026-08-15T13:26:05.181Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"} gitlab # [ 337.390953] nixbot[3556]: {"ts": "2026-08-15T13:26:05.182Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"} gitlab # [ 337.396202] nixbot[3556]: {"ts": "2026-08-15T13:26:05.183Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"} gitlab # [ 337.404379] nixbot[3556]: {"ts": "2026-08-15T13:26:05.184Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"} gitlab # [ 337.406854] nixbot[3556]: {"ts": "2026-08-15T13:26:05.188Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"} gitlab # [ 337.409011] nixbot[3556]: {"ts": "2026-08-15T13:26:05.189Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"} gitlab # [ 337.412234] nixbot[3556]: {"ts": "2026-08-15T13:26:05.191Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"} gitlab # [ 337.414543] nixbot[3556]: {"ts": "2026-08-15T13:26:05.195Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"} gitlab # [ 337.417523] nixbot[3556]: {"ts": "2026-08-15T13:26:05.196Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"} gitlab # [ 337.420199] nixbot[3556]: {"ts": "2026-08-15T13:26:05.198Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"} gitlab # [ 337.423600] nixbot[3556]: {"ts": "2026-08-15T13:26:05.199Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"} gitlab # [ 337.740101] nixbot[3556]: INFO: Started server process [3556] gitlab # [ 337.741071] nixbot[3556]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit) gitlab # [ 337.761235] nixbot[3556]: INFO: ::ffff:127.0.0.1:37002 - "GET /health HTTP/1.1" 200 OK gitlab # [ 337.768448] systemd[1]: Started nixbot CI service. gitlab # [ 337.771785] systemd[1]: Reached target Multi-User System. gitlab # [ 337.772670] systemd[1]: Startup finished in 805ms (kernel) + 4.728s (initrd) + 5min 32.239s (userspace) = 5min 37.772s. gitlab: (finished: waiting for unit nixbot.service, in 2.15 seconds) gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health gitlab # [ 338.613251] nixbot[3556]: INFO: ::ffff:127.0.0.1:37018 - "GET /health HTTP/1.1" 200 OK gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.06 seconds) subtest: project discovered and webhook registered gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 338.816128] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "curl/8.21.0" 145 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.20 seconds) gitlab # [ 339.051935] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:06 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1290 gitlab # [ 339.054399] nixbot[3556]: {"ts": "2026-08-15T13:26:06.884Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 339.081703] nixbot[3556]: {"ts": "2026-08-15T13:26:06.912Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1} gitlab # [ 339.154894] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:06 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 58 gitlab # [ 339.161525] nixbot[3556]: {"ts": "2026-08-15T13:26:06.987Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 339.168482] nixbot[3556]: {"ts": "2026-08-15T13:26:06.991Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"} gitlab # [ 339.352353] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 919 "" "python-httpx/0.28.1" 180 gitlab # [ 339.359634] nixbot[3556]: {"ts": "2026-08-15T13:26:07.178Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""} gitlab # [ 339.566720] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 195 gitlab # [ 339.574097] nixbot[3556]: {"ts": "2026-08-15T13:26:07.399Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 339.654115] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "python-httpx/0.28.1" 69 gitlab # [ 339.671622] nixbot[3556]: {"ts": "2026-08-15T13:26:07.493Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""} gitlab # [ 339.740526] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 919 "" "python-httpx/0.28.1" 69 gitlab # [ 339.747688] nixbot[3556]: {"ts": "2026-08-15T13:26:07.572Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks gitlab # [ 339.828992] gitaly[904]: time="2026-08-15T13:26:07.655Z" level=info msg="finished unary call with code OK" catfile.duration_ms=7 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=7 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKSYRH9K5S364C8JHPR8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-08-15T13:26:37.027" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:07.628" grpc.time_ms=26.262 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.872666] gitaly[904]: time="2026-08-15T13:26:07.673Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKSYRH9K5S364C8JHPR8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:37.071" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:07.672" grpc.time_ms=0.971 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.899368] gitaly[904]: time="2026-08-15T13:26:07.690Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKSYRH9K5S364C8JHPR8C grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-08-15T13:26:17.009" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:07.680" grpc.time_ms=9.509 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 339.923310] nixbot[3556]: {"ts": "2026-08-15T13:26:07.727Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""} gitlab # [ 339.927605] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 149 gitlab # [ 340.054957] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:07 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 154 gitlab # [ 340.062724] nixbot[3556]: {"ts": "2026-08-15T13:26:07.886Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""} gitlab # [ 340.071249] nixbot[3556]: {"ts": "2026-08-15T13:26:07.890Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null} gitlab # [ 340.176205] gitaly[904]: time="2026-08-15T13:26:08.004Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKT9S73WPH7ZTJ2HAETD6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=196 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:07.994" grpc.time_ms=9.294 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.195151] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 76 gitlab # [ 340.293587] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 882 "" "curl/8.21.0" 384 gitlab # [ 340.295992] gitlab-workhorse[820]: time="2026-08-15T13:26:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=113 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.48 seconds) (finished: subtest: project discovered and webhook registered, in 1.68 seconds) subtest: push delivers the webhook gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master gitlab # [ 340.298030] gitaly[904]: time="2026-08-15T13:26:08.124Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTC8XW8MH3E3MMRSEQMB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.112" pid=904 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc gitlab # [ 340.311424] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 118 gitlab # [ 340.313603] gitaly[904]: time="2026-08-15T13:26:08.125Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTC8XW8MH3E3MMRSEQMB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.112" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.327690] gitaly[904]: time="2026-08-15T13:26:08.125Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=588 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTC8XW8MH3E3MMRSEQMB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.112" grpc.time_ms=12.417 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.419096] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 36 gitlab # [ 340.430184] gitaly[904]: time="2026-08-15T13:26:08.260Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTG7VMG1RBHFKQZX5VZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.234" pid=904 remote_ip="2001:db8:1::1" request_sha=375258203fa868d1dbbc6b19c7dda193f3aa6a2c response_bytes=228 span.kind=server system=grpc gitlab # [ 340.454128] gitaly[904]: time="2026-08-15T13:26:08.260Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTG7VMG1RBHFKQZX5VZZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.234" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.481326] gitlab-workhorse[820]: time="2026-08-15T13:26:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=228 gitlab # [ 340.485620] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 228 "" "git/2.55.0" 126 gitlab # [ 340.488738] gitaly[904]: time="2026-08-15T13:26:08.263Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=21 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=23 command.spawn_token_fork_ms=0 command.system_time_ms=10 command.user_time_ms=11 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTG7VMG1RBHFKQZX5VZZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.234" grpc.time_ms=26.929 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.551983] gitaly[904]: time="2026-08-15T13:26:08.382Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTNSN1KEYVQZC48BJFPR grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=192 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.370" grpc.time_ms=11.894 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.581193] gitaly[904]: time="2026-08-15T13:26:08.394Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=4 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTKVH5E2KF6MV8AMDQ0A grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=246 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.386" grpc.time_ms=7.353 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # Enumerating objects: 5, done. gitlab # [ 340.601610] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # Counting objects: 20% (1/5)[ 340.604647] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 143 gitlab # Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done. gitlab # Delta compression using up to 4 threads gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done. gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 274 bytes | 274.00 KiB/s, done. gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0) gitlab # [ 340.627378] gitaly[904]: time="2026-08-15T13:26:08.458Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTR0FWCNBG5QVPKD5QCH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.446" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 340.644373] gitaly[904]: time="2026-08-15T13:26:08.458Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTR0FWCNBG5QVPKD5QCH grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.446" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.662778] gitlab-workhorse[820]: time="2026-08-15T13:26:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 340.665101] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 74 gitlab # [ 340.667356] gitaly[904]: time="2026-08-15T13:26:08.458Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTR0FWCNBG5QVPKD5QCH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.446" grpc.time_ms=11.79 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.769550] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 69 gitlab # [ 340.772182] gitaly[904]: time="2026-08-15T13:26:08.600Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTWJ4SFV10G34JKGFMTW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.591" grpc.time_ms=7.448 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.814256] gitaly[904]: time="2026-08-15T13:26:08.644Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=301 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=5 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-08-15T13:26:38.035" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-08-15T13:26:08.635" grpc.time_ms=9.002 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 340.839253] gitaly[904]: time="2026-08-15T13:26:08.669Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M02SKTT07FY64FC53972H4DM duration_ms=102ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/allowed" gitlab # [ 340.858315] gitaly[904]: time="2026-08-15T13:26:08.689Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTYV4VVCSEQ17K6407ZN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.665" pid=904 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc gitlab # [ 340.876074] gitaly[904]: time="2026-08-15T13:26:08.689Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTYV4VVCSEQ17K6407ZN grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.665" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.897911] gitlab-workhorse[820]: time="2026-08-15T13:26:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=209 gitlab # [ 340.900619] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 86 gitlab # [ 340.903241] gitaly[904]: time="2026-08-15T13:26:08.689Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1102 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=4 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTYV4VVCSEQ17K6407ZN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.665" grpc.time_ms=24.193 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 340.922976] gitaly[904]: time="2026-08-15T13:26:08.707Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M02SKTT07FY64FC53972H4DM duration_ms=36ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/pre_receive" gitlab # [ 340.927470] gitaly[904]: time="2026-08-15T13:26:08.708Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:08.566" grpc.time_ms=141.615 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 340.940608] gitaly[904]: time="2026-08-15T13:26:08.735Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:08.735" grpc.time_ms=0.728 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 340.954541] gitaly[904]: time="2026-08-15T13:26:08.784Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:08.783" grpc.time_ms=1.222 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 340.978750] gitaly[904]: time="2026-08-15T13:26:08.809Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:08.808" grpc.time_ms=0.49 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # [ 341.031865] gitaly[904]: time="2026-08-15T13:26:08.862Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:38.261" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:08.861" grpc.time_ms=0.559 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.078299] gitaly[904]: time="2026-08-15T13:26:08.908Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M02SKTT07FY64FC53972H4DM duration_ms=66ns method=POST pid=904 status=200 url="http://unix/api/v4/internal/post_receive" gitlab # [ 341.084368] gitaly[904]: time="2026-08-15T13:26:08.909Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:08.842" grpc.time_ms=67.181 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab # To http://gitlab/root/test-flake.git gitlab # [ 341.103125] gitaly[904]: time="2026-08-15T13:26:08.910Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=591 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-15T13:26:38.292" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:08.893" grpc.time_ms=16.685 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # 4b2631e..b0d1904 master -> master[ 341.125571] gitlab-workhorse[820]: time="2026-08-15T13:26:08Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=52 gitlab # gitlab # [ 341.128064] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:08 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 472 gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 0.85 seconds) gitlab # [ 341.130974] gitaly[904]: time="2026-08-15T13:26:08.920Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM diskcache=d4e78ca8-dd92-4c19-8027-bf07c43efd4f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.515" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab: must succeed: git -C /tmp/test-flake rev-parse master gitlab # [ 341.144215] gitaly[904]: time="2026-08-15T13:26:08.920Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=141 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13307 command.oublock=32 command.real_time_ms=399 command.spawn_token_fork_ms=0 command.system_time_ms=87 command.user_time_ms=54 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=606 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=58 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:08.515" grpc.time_ms=405.597 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.01 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 341.161197] gitaly[904]: time="2026-08-15T13:26:08.942Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:26:18.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:08.934" grpc.time_ms=7.828 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.178672] gitaly[904]: time="2026-08-15T13:26:08.960Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=2 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:26:18.051" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:08.952" grpc.time_ms=8.137 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.224451] gitaly[904]: time="2026-08-15T13:26:09.055Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-15T13:26:19.004" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:09.054" grpc.time_ms=0.465 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.08 seconds) gitlab # [ 341.243761] nixbot[3556]: INFO: ::ffff:127.0.0.1:47440 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab # [ 341.248207] gitaly[904]: time="2026-08-15T13:26:09.074Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=3 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-15T13:26:19.016" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:09.067" grpc.time_ms=7.573 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.281945] gitaly[904]: time="2026-08-15T13:26:09.112Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:09.111" grpc.time_ms=0.43 pid=904 span.kind=server system=grpc gitlab # [ 341.314966] gitaly[904]: time="2026-08-15T13:26:09.145Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:26:09.144" grpc.time_ms=0.377 pid=904 span.kind=server system=grpc gitlab # [ 341.325450] gitaly[904]: time="2026-08-15T13:26:09.152Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 diskcache=6452700b-e6c0-4b2a-92aa-1821a6443127 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:26:19.030" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:09.081" pid=904 ref=refs/keep-around/4b2631ed070ad5751ddf024b91e967d30ab33e37 remote_ip="2001:db8:1::1" revision=4b2631ed070ad5751ddf024b91e967d30ab33e37 span.kind=server system=grpc username=root gitlab # [ 341.337017] gitaly[904]: time="2026-08-15T13:26:09.153Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5199 command.oublock=8 command.real_time_ms=65 command.spawn_token_fork_ms=1 command.system_time_ms=32 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:26:19.030" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:26:09.081" grpc.time_ms=71.762 pid=904 ref=refs/keep-around/4b2631ed070ad5751ddf024b91e967d30ab33e37 remote_ip="2001:db8:1::1" revision=4b2631ed070ad5751ddf024b91e967d30ab33e37 span.kind=server system=grpc username=root gitlab # [ 341.351874] gitaly[904]: time="2026-08-15T13:26:09.161Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:39.060" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:09.160" grpc.time_ms=0.518 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 341.542892] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:09 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 584 gitlab # [ 341.546032] nixbot[3556]: {"ts": "2026-08-15T13:26:09.375Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 341.557616] nixbot[3556]: {"ts": "2026-08-15T13:26:09.388Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4b2631ed070a-0"} gitlab # [ 341.673763] systemd[1]: Started Nix Daemon. gitlab # [ 341.801681] nix-daemon[3765]: accepted connection from pid 3759, user nixbot gitlab # [ 341.940668] nix-daemon[3765]: accepted connection from pid 3772, user nixbot gitlab # [ 341.965961] nixbot[3556]: {"ts": "2026-08-15T13:26:09.796Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4b2631ed070a-0", "argv0": "bwrap"} gitlab # [ 342.204074] nix-daemon[3765]: accepted connection from pid 3782, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 342.301682] nixbot[3556]: INFO: ::ffff:127.0.0.1:47442 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 342.315868] nix-daemon[3765]: accepted connection from pid 3792, user nixbot gitlab # [ 342.338776] nix-daemon[3765]: accepted connection from pid 3782, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 343.358148] nixbot[3556]: INFO: ::ffff:127.0.0.1:47444 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 344.413648] nixbot[3556]: INFO: ::ffff:127.0.0.1:47446 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 345.468623] nixbot[3556]: INFO: ::ffff:127.0.0.1:47460 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 346.525025] nixbot[3556]: INFO: ::ffff:127.0.0.1:47474 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 347.149805] gitaly[904]: time="2026-08-15T13:26:14.979Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM12AWCH1W51TY7GB76AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:44.377" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:14.977" grpc.time_ms=1.22 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.438272] gitaly[904]: time="2026-08-15T13:26:15.268Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=3 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SM12AWCH1W51TY7GB76AY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:26:25.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:15.246" grpc.time_ms=21.211 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 347.592123] nixbot[3556]: INFO: ::ffff:127.0.0.1:47480 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 347.629308] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 670 "" "python-httpx/0.28.1" 599 gitlab # [ 347.639061] nixbot[3556]: {"ts": "2026-08-15T13:26:15.460Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 347.694966] gitaly[904]: time="2026-08-15T13:26:15.525Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM1N8QYDXQ2KDXXWC8NVW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:45.023" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:15.524" grpc.time_ms=0.94 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.766957] gitaly[904]: time="2026-08-15T13:26:15.597Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SM1N8QYDXQ2KDXXWC8NVW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:26:25.274" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:15.585" grpc.time_ms=11.831 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 347.884234] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:15 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 244 gitlab # [ 347.893674] nixbot[3556]: {"ts": "2026-08-15T13:26:15.711Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 347.999539] gitaly[904]: time="2026-08-15T13:26:15.830Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM1XZFK87NEJJTZ9WVN29 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:26:45.228" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:26:15.828" grpc.time_ms=1.099 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 348.076680] gitaly[904]: time="2026-08-15T13:26:15.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SM1XZFK87NEJJTZ9WVN29 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:26:25.217" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:26:15.888" grpc.time_ms=18.306 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 348.177609] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:16 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 263 gitlab # [ 348.185811] nixbot[3556]: {"ts": "2026-08-15T13:26:16.009Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 348.343916] nix-daemon[3765]: accepted connection from pid 3835, user nixbot gitlab # [ 348.521603] gitaly[904]: time="2026-08-15T13:26:16.351Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=6 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SM2DZ6NE71KGC67XKSQ2G grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.332" grpc.time_ms=18.417 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.544161] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:16 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 96 gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 348.657762] nixbot[3556]: INFO: ::ffff:127.0.0.1:47490 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 348.663028] gitaly[904]: time="2026-08-15T13:26:16.492Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2H2X7HSZGBT472JE1NY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.471" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 348.686349] gitaly[904]: time="2026-08-15T13:26:16.492Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2H2X7HSZGBT472JE1NY grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.471" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.708511] gitlab-workhorse[820]: time="2026-08-15T13:26:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 348.711048] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 140 gitlab # [ 348.713757] gitaly[904]: time="2026-08-15T13:26:16.493Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=586 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2H2X7HSZGBT472JE1NY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.471" grpc.time_ms=22.434 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.793164] gitaly[904]: time="2026-08-15T13:26:16.623Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2NKA8YM88ECGR9TPZ93 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.594" pid=904 remote_ip="2001:db8:1::1" request_sha=7b6e38bbc1043ac0578c13a78cb56d7857f278a2 response_bytes=338 span.kind=server system=grpc gitlab # [ 348.821888] gitaly[904]: time="2026-08-15T13:26:16.623Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2NKA8YM88ECGR9TPZ93 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.594" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.848696] gitlab-workhorse[820]: time="2026-08-15T13:26:16Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=338 gitlab # [ 348.851574] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:26:16 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 338 "" "git/2.55.0" 125 gitlab # [ 348.854723] gitaly[904]: time="2026-08-15T13:26:16.624Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=24 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1131 command.oublock=0 command.real_time_ms=26 command.spawn_token_fork_ms=0 command.system_time_ms=19 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M02SM2NKA8YM88ECGR9TPZ93 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:26:16.594" grpc.time_ms=29.435 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 348.953503] nix-daemon[3765]: accepted connection from pid 3868, user nixbot gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 349.718702] nixbot[3556]: INFO: ::ffff:127.0.0.1:47500 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 350.770783] nixbot[3556]: INFO: ::ffff:127.0.0.1:47504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 351.825964] nixbot[3556]: INFO: ::ffff:127.0.0.1:60416 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 352.880874] nixbot[3556]: INFO: ::ffff:127.0.0.1:60424 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 353.937821] nixbot[3556]: INFO: ::ffff:127.0.0.1:60434 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 354.992916] nixbot[3556]: INFO: ::ffff:127.0.0.1:60440 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 356.046782] nixbot[3556]: INFO: ::ffff:127.0.0.1:60450 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 357.103728] nixbot[3556]: INFO: ::ffff:127.0.0.1:60458 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 358.159901] nixbot[3556]: INFO: ::ffff:127.0.0.1:60468 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 359.214907] nixbot[3556]: INFO: ::ffff:127.0.0.1:60474 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 360.269222] nixbot[3556]: INFO: ::ffff:127.0.0.1:60480 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 361.322591] nixbot[3556]: INFO: ::ffff:127.0.0.1:33114 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 362.377871] nixbot[3556]: INFO: ::ffff:127.0.0.1:33130 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 363.433063] nixbot[3556]: INFO: ::ffff:127.0.0.1:33144 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 364.486172] nixbot[3556]: INFO: ::ffff:127.0.0.1:33152 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 365.538918] nixbot[3556]: INFO: ::ffff:127.0.0.1:33160 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 366.592869] nixbot[3556]: INFO: ::ffff:127.0.0.1:33172 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 367.645025] nixbot[3556]: INFO: ::ffff:127.0.0.1:33178 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 368.699788] nixbot[3556]: INFO: ::ffff:127.0.0.1:33186 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 369.754801] nixbot[3556]: INFO: ::ffff:127.0.0.1:33196 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 370.807666] nixbot[3556]: INFO: ::ffff:127.0.0.1:33198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 371.863071] nixbot[3556]: INFO: ::ffff:127.0.0.1:34792 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 372.915811] nixbot[3556]: INFO: ::ffff:127.0.0.1:34802 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 373.970574] nixbot[3556]: INFO: ::ffff:127.0.0.1:34816 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 374.707790] sidekiq-cluster[3536]: 2026-08-15T13:26:42.537Z pid=3536 tid=yns INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3536, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>433008640} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 374.982248] sidekiq-cluster[3536]: W, [2026-08-15T13:26:42.812760 #3536] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`. gitlab # [ 375.025983] nixbot[3556]: INFO: ::ffff:127.0.0.1:34822 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 375.046632] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:26:42.877Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 376.081361] nixbot[3556]: INFO: ::ffff:127.0.0.1:34838 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 377.134587] nixbot[3556]: INFO: ::ffff:127.0.0.1:34850 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 378.190485] nixbot[3556]: INFO: ::ffff:127.0.0.1:34854 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 378.524393] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:26:46.353Z","message":"GitLab reliable fetch activated!"} gitlab # [ 378.529318] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:26:46.354Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 379.245579] nixbot[3556]: INFO: ::ffff:127.0.0.1:34856 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 380.297547] nixbot[3556]: INFO: ::ffff:127.0.0.1:34858 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 381.350132] nixbot[3556]: INFO: ::ffff:127.0.0.1:46462 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 382.403996] nixbot[3556]: INFO: ::ffff:127.0.0.1:46474 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 383.456071] nixbot[3556]: INFO: ::ffff:127.0.0.1:46482 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 384.510979] nixbot[3556]: INFO: ::ffff:127.0.0.1:46486 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 385.562752] nixbot[3556]: INFO: ::ffff:127.0.0.1:46490 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 386.618183] nixbot[3556]: INFO: ::ffff:127.0.0.1:46492 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 387.670107] nixbot[3556]: INFO: ::ffff:127.0.0.1:46496 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 388.725095] nixbot[3556]: INFO: ::ffff:127.0.0.1:46504 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 389.780277] nixbot[3556]: INFO: ::ffff:127.0.0.1:46514 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 390.832746] nixbot[3556]: INFO: ::ffff:127.0.0.1:59918 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 391.887549] nixbot[3556]: INFO: ::ffff:127.0.0.1:59934 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 392.942760] nixbot[3556]: INFO: ::ffff:127.0.0.1:59938 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 393.997408] nixbot[3556]: INFO: ::ffff:127.0.0.1:59946 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 395.049618] nixbot[3556]: INFO: ::ffff:127.0.0.1:59948 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 396.103714] nixbot[3556]: INFO: ::ffff:127.0.0.1:59956 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 397.155814] nixbot[3556]: INFO: ::ffff:127.0.0.1:59964 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 398.207998] nixbot[3556]: INFO: ::ffff:127.0.0.1:59976 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 399.260950] nixbot[3556]: INFO: ::ffff:127.0.0.1:59988 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 400.315173] nixbot[3556]: INFO: ::ffff:127.0.0.1:59992 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 401.367756] nixbot[3556]: INFO: ::ffff:127.0.0.1:40198 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 402.422654] nixbot[3556]: INFO: ::ffff:127.0.0.1:40210 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 403.476695] nixbot[3556]: INFO: ::ffff:127.0.0.1:40214 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 404.530565] nixbot[3556]: INFO: ::ffff:127.0.0.1:40222 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 405.584676] nixbot[3556]: INFO: ::ffff:127.0.0.1:40228 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 406.640933] nixbot[3556]: INFO: ::ffff:127.0.0.1:40232 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 407.695764] nixbot[3556]: INFO: ::ffff:127.0.0.1:40238 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 408.747077] nixbot[3556]: INFO: ::ffff:127.0.0.1:40248 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 409.801968] nixbot[3556]: INFO: ::ffff:127.0.0.1:40254 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 410.854474] nixbot[3556]: INFO: ::ffff:127.0.0.1:42938 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 411.906569] nixbot[3556]: INFO: ::ffff:127.0.0.1:42940 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 412.963000] nixbot[3556]: INFO: ::ffff:127.0.0.1:42948 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 414.016698] nixbot[3556]: INFO: ::ffff:127.0.0.1:42958 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 415.070384] nixbot[3556]: INFO: ::ffff:127.0.0.1:42964 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 416.030114] postgres[4083]: [4083] ERROR: relation "table_without_models" does not exist gitlab # [ 416.030477] postgres[4083]: [4083] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id') gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 416.124211] nixbot[3556]: INFO: ::ffff:127.0.0.1:42978 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 417.177524] nixbot[3556]: INFO: ::ffff:127.0.0.1:42992 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 418.230223] nixbot[3556]: INFO: ::ffff:127.0.0.1:42996 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 419.281829] nixbot[3556]: INFO: ::ffff:127.0.0.1:43006 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 420.333937] nixbot[3556]: INFO: ::ffff:127.0.0.1:43018 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 420.811015] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:28.640Z","message":"Booted Rails 7.2.3.1 application in production environment"} gitlab # [ 420.816629] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:28.640Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"} gitlab # [ 420.823186] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:28.641Z","message":"See LICENSE and the LGPL-3.0 for licensing details."} gitlab # [ 420.828785] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:28.641Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"} gitlab # [ 420.834943] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:28.644Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 421.389953] nixbot[3556]: INFO: ::ffff:127.0.0.1:60854 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.05 seconds) gitlab # [ 422.442402] nixbot[3556]: INFO: ::ffff:127.0.0.1:60862 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 423.432355] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:31.261Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"df56dd344e80f2ccb4a8ebc9","created_at":"2026-08-15T13:26:00.370Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:00.433Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-df56dd344e80f2ccb4a8ebc9: start","job_status":"start","queue_duration_s":90.827723,"scheduling_latency_s":90.827837} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 423.506774] nixbot[3556]: INFO: ::ffff:127.0.0.1:60872 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab # [ 423.811995] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:31.641Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"df56dd344e80f2ccb4a8ebc9","created_at":"2026-08-15T13:26:00.370Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:00.433Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-df56dd344e80f2ccb4a8ebc9: done: 0.380985 sec","job_status":"done","queue_duration_s":90.827723,"scheduling_latency_s":90.827837,"redis_calls":23,"redis_duration_s":0.01042,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.00559,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002819,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002011,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.023,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.198056,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.19922004,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.380985,"completed_at":"2026-08-15T13:27:31.641Z","load_balancing_strategy":"primary","db_duration_s":0.108313,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 423.931882] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:31.644Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5fa98f6292df37a6c4909ad3","created_at":"2026-08-15T13:26:00.455Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:00.465Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5fa98f6292df37a6c4909ad3: start","job_status":"start","queue_duration_s":91.179463,"scheduling_latency_s":91.179503} gitlab # [ 424.040641] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:31.870Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"5fa98f6292df37a6c4909ad3","created_at":"2026-08-15T13:26:00.455Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:00.465Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-5fa98f6292df37a6c4909ad3: done: 0.225252 sec","job_status":"done","queue_duration_s":91.179463,"scheduling_latency_s":91.179503,"redis_calls":14,"redis_duration_s":0.006987,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001096,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003655,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002236,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.172686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.17214518,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.225252,"completed_at":"2026-08-15T13:27:31.869Z","load_balancing_strategy":"primary","db_duration_s":0.040359,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.133421] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:31.873Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"5e06df8b2e8494ab2c67f5f9","created_at":"2026-08-15T13:26:01.141Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.149Z","job_size_bytes":5,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-5e06df8b2e8494ab2c67f5f9: start","job_status":"start","queue_duration_s":90.723576,"scheduling_latency_s":90.72362} gitlab # [ 424.298196] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.128Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"5e06df8b2e8494ab2c67f5f9","created_at":"2026-08-15T13:26:01.141Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.149Z","job_size_bytes":5,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-5e06df8b2e8494ab2c67f5f9: done: 0.254868 sec","job_status":"done","queue_duration_s":90.723576,"scheduling_latency_s":90.72362,"redis_calls":12,"redis_duration_s":0.004352,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001205,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002461,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000686,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.198022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005743,"duration_s":0.254868,"completed_at":"2026-08-15T13:27:32.128Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.047149,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 424.403885] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.130Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c6f8de072caca315f937a19d","created_at":"2026-08-15T13:26:01.184Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.190Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c6f8de072caca315f937a19d: start","job_status":"start","queue_duration_s":90.940423,"scheduling_latency_s":90.940451} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 424.565921] nixbot[3556]: INFO: ::ffff:127.0.0.1:60876 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 424.622069] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.450Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"c6f8de072caca315f937a19d","created_at":"2026-08-15T13:26:01.184Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.190Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-c6f8de072caca315f937a19d: done: 0.320191 sec","job_status":"done","queue_duration_s":90.940423,"scheduling_latency_s":90.940451,"redis_calls":11,"redis_duration_s":0.003861,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000966,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002196,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000699,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.268,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.268,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.046,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.207633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069632,"duration_s":0.320191,"completed_at":"2026-08-15T13:27:32.450Z","load_balancing_strategy":"primary","db_duration_s":0.116701,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.715590] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.453Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"87aa769d1f3dfa8cbb7990f0","created_at":"2026-08-15T13:26:01.193Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.199Z","job_size_bytes":89,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-87aa769d1f3dfa8cbb7990f0: start","job_status":"start","queue_duration_s":91.253991,"scheduling_latency_s":91.254018} gitlab # [ 424.736200] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.517Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"87aa769d1f3dfa8cbb7990f0","created_at":"2026-08-15T13:26:01.193Z","trace_propagation_headers":{"sentry-trace":"db4404b867564ec9b59f987d1369efc9-a7f4759bb44849de","baggage":"sentry-trace_id=db4404b867564ec9b59f987d1369efc9,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKGTEATY7ZXMX2ECEKRA9","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:01.199Z","job_size_bytes":89,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-87aa769d1f3dfa8cbb7990f0: done: 0.064554 sec","job_status":"done","queue_duration_s":91.253991,"scheduling_latency_s":91.254018,"redis_calls":11,"redis_duration_s":0.003882,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.001077,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002115,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00069,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00089032,"duration_s":0.064554,"completed_at":"2026-08-15T13:27:32.517Z","load_balancing_strategy":"primary","db_duration_s":0.00901,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 424.783529] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.522Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4b2631ed070ad5751ddf024b91e967d30ab33e37 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cf0b066a4dfb80b390835634","created_at":"2026-08-15T13:26:03.240Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-a642de6974424473","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:26:03.272Z","job_size_bytes":185,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cf0b066a4dfb80b390835634: start","job_status":"start","queue_duration_s":89.249633,"scheduling_latency_s":89.249682} gitlab # [ 424.800559] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:32.560Z","message":"changes: \"0000000000000000000000000000000000000000 4b2631ed070ad5751ddf024b91e967d30ab33e37 refs/heads/master\\n\""} gitlab # [ 424.806277] gitaly[904]: time="2026-08-15T13:27:32.636Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:42.007" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:32.617" grpc.time_ms=18.597 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.832710] gitaly[904]: time="2026-08-15T13:27:32.663Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:27:42.035" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:32.646" grpc.time_ms=16.737 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 424.937407] gitaly[904]: time="2026-08-15T13:27:32.767Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=10 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-15T13:27:42.137" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:32.747" grpc.time_ms=19.799 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.019388] gitaly[904]: time="2026-08-15T13:27:32.850Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:32.849" grpc.time_ms=0.791 pid=904 span.kind=server system=grpc gitlab # [ 425.068114] gitaly[904]: time="2026-08-15T13:27:32.898Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:32.898" grpc.time_ms=0.533 pid=904 span.kind=server system=grpc gitlab # [ 425.097236] gitaly[904]: time="2026-08-15T13:27:32.911Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 diskcache=beb0569b-9094-4153-839d-50c28e83ecbc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:27:42.180" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:32.790" pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 425.119092] gitaly[904]: time="2026-08-15T13:27:32.912Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=132 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5119 command.oublock=8 command.real_time_ms=113 command.spawn_token_fork_ms=1 command.system_time_ms=76 command.user_time_ms=56 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:27:42.180" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:32.790" grpc.time_ms=121.185 pid=904 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root gitlab # [ 425.140915] gitaly[904]: time="2026-08-15T13:27:32.935Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-08-15T13:27:42.004" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:32.935" grpc.time_ms=0.329 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.154938] gitaly[904]: time="2026-08-15T13:27:32.966Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-08-15T13:27:42.011" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:32.941" grpc.time_ms=25.071 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.311472] gitaly[904]: time="2026-08-15T13:27:33.141Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=313 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-08-15T13:28:03.029" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:33.129" grpc.time_ms=12.226 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.376643] gitaly[904]: time="2026-08-15T13:27:33.207Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-15T13:27:43.005" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:33.206" grpc.time_ms=0.741 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.409814] gitaly[904]: time="2026-08-15T13:27:33.222Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=315 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-15T13:27:43.010" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-08-15T13:27:33.211" grpc.time_ms=10.845 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.441277] gitaly[904]: time="2026-08-15T13:27:33.265Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:43.048" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:33.248" grpc.time_ms=16.718 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.462829] gitaly[904]: time="2026-08-15T13:27:33.292Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:43.075" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:33.276" grpc.time_ms=16.123 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 425.511911] gitaly[904]: time="2026-08-15T13:27:33.342Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=13 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-15T13:28:03.215" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:33.315" grpc.time_ms=26.657 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 425.633412] nixbot[3556]: INFO: ::ffff:127.0.0.1:60890 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 426.092452] gitaly[904]: time="2026-08-15T13:27:33.921Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-15T13:27:43.009" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:33.920" grpc.time_ms=1.469 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.132300] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:33.955Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4b2631ed070ad5751ddf024b91e967d30ab33e37 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"cf0b066a4dfb80b390835634","created_at":"2026-08-15T13:26:03.240Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-a642de6974424473","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:26:03.272Z","job_size_bytes":185,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-cf0b066a4dfb80b390835634: done: 1.434174 sec","job_status":"done","queue_duration_s":89.249633,"scheduling_latency_s":89.249682,"gitaly_calls":13,"gitaly_duration_s":0.350899,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.064127,"redis_read_bytes":2103,"redis_write_bytes":21749,"redis_cache_calls":15,"redis_cache_duration_s":0.006714,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.008639,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.002738,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7837,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.005722,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.02696,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.013354,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.089,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.319,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.367,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.089,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.939218,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00121316,"exclusive_lock_hold_duration_s":0.05234208,"duration_s":1.434174,"completed_at":"2026-08-15T13:27:33.955Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.157344,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.220795] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:33.957Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4b2631ed070ad5751ddf024b91e967d30ab33e37 b0d1904a3636a352e89f8292bcc2601f40d8477e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4f2fbb36904d9197e5e943e4","created_at":"2026-08-15T13:26:08.885Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-d265f1deafbd4d64","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:26:08.887Z","job_size_bytes":185,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4f2fbb36904d9197e5e943e4: start","job_status":"start","queue_duration_s":85.070455,"scheduling_latency_s":85.070478} gitlab # [ 426.239759] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:33.971Z","message":"changes: \"4b2631ed070ad5751ddf024b91e967d30ab33e37 b0d1904a3636a352e89f8292bcc2601f40d8477e refs/heads/master\\n\""} gitlab # [ 426.242686] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.033Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"11308e50fce09e13b0ae13e2","created_at":"2026-08-15T13:27:34.031Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"d718ec66d6a3a3228fd6a6b6","job_size_bytes":46,"pid":3536,"job_status":"deduplicated","message":"ProjectCacheWorker JID-11308e50fce09e13b0ae13e2: deduplicated: until executing","deduplication.type":"until executing"} gitlab # [ 426.259678] gitaly[904]: time="2026-08-15T13:27:34.007Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:43.059" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:33.989" grpc.time_ms=17.317 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.274633] gitaly[904]: time="2026-08-15T13:27:34.028Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-08-15T13:27:44.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:34.012" grpc.time_ms=15.824 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.288637] gitaly[904]: time="2026-08-15T13:27:34.064Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-08-15T13:27:44.037" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:34.047" grpc.time_ms=16.367 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.302034] gitaly[904]: time="2026-08-15T13:27:34.088Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-08-15T13:28:04.069" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:34.069" grpc.time_ms=18.799 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.317130] gitaly[904]: time="2026-08-15T13:27:34.111Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-08-15T13:27:44.090" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-08-15T13:27:34.100" grpc.time_ms=10.577 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.330372] gitaly[904]: time="2026-08-15T13:27:34.135Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:44.111" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:34.121" grpc.time_ms=13.779 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.534647] gitaly[904]: time="2026-08-15T13:27:34.365Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-08-15T13:27:44.228" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:34.238" grpc.time_ms=126.616 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 426.578813] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.408Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4b2631ed070ad5751ddf024b91e967d30ab33e37 b0d1904a3636a352e89f8292bcc2601f40d8477e refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"4f2fbb36904d9197e5e943e4","created_at":"2026-08-15T13:26:08.885Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-d265f1deafbd4d64","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-08-15T13:26:08.887Z","job_size_bytes":185,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-4f2fbb36904d9197e5e943e4: done: 0.450274 sec","job_status":"done","queue_duration_s":85.070455,"scheduling_latency_s":85.070478,"gitaly_calls":7,"gitaly_duration_s":0.239834,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.026928,"redis_read_bytes":500,"redis_write_bytes":11940,"redis_cache_calls":9,"redis_cache_duration_s":0.004519,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.001379,"redis_queues_read_bytes":8,"redis_queues_write_bytes":5717,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003564,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007912,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009554,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.033,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.028,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.028,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.033,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.290926,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00079318,"exclusive_lock_hold_duration_s":0.02701698,"duration_s":0.450274,"completed_at":"2026-08-15T13:27:34.408Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034258,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 426.632667] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.411Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"854ce1c1e4f9133dfabb72e0","created_at":"2026-08-15T13:26:09.167Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.179Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-854ce1c1e4f9133dfabb72e0: start","job_status":"start","queue_duration_s":85.231806,"scheduling_latency_s":85.231849} gitlab # [ 426.698593] nixbot[3556]: INFO: ::ffff:127.0.0.1:60904 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 426.756789] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.586Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"854ce1c1e4f9133dfabb72e0","created_at":"2026-08-15T13:26:09.167Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.179Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-854ce1c1e4f9133dfabb72e0: done: 0.175066 sec","job_status":"done","queue_duration_s":85.231806,"scheduling_latency_s":85.231849,"redis_calls":16,"redis_duration_s":0.006866000000000001,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.003504,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002533,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000829,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.122024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00083414,"duration_s":0.175066,"completed_at":"2026-08-15T13:27:34.586Z","load_balancing_strategy":"primary","db_duration_s":0.041098,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 426.846370] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.589Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"04d78f0db23859fa521f550a","created_at":"2026-08-15T13:26:09.180Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.187Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-04d78f0db23859fa521f550a: start","job_status":"start","queue_duration_s":85.402081,"scheduling_latency_s":85.402114} gitlab # [ 426.942272] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.772Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"04d78f0db23859fa521f550a","created_at":"2026-08-15T13:26:09.180Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.187Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-04d78f0db23859fa521f550a: done: 0.182713 sec","job_status":"done","queue_duration_s":85.402081,"scheduling_latency_s":85.402114,"redis_calls":21,"redis_duration_s":0.007693,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.003555,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002021,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.002117,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.118377,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068004,"duration_s":0.182713,"completed_at":"2026-08-15T13:27:34.772Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.051666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 427.028106] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:34.774Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56c176e6502c945b1b274e32","created_at":"2026-08-15T13:26:09.342Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.350Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56c176e6502c945b1b274e32: start","job_status":"start","queue_duration_s":85.424807,"scheduling_latency_s":85.424899} gitlab # [ 427.216257] gitaly[904]: time="2026-08-15T13:27:35.045Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:35.044" grpc.time_ms=0.747 pid=904 span.kind=server system=grpc gitlab # [ 427.268037] gitaly[904]: time="2026-08-15T13:27:35.098Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:35.097" grpc.time_ms=0.718 pid=904 span.kind=server system=grpc gitlab # [ 427.316312] gitaly[904]: time="2026-08-15T13:27:35.146Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:35.146" grpc.time_ms=0.568 pid=904 span.kind=server system=grpc gitlab # [ 427.346396] gitaly[904]: time="2026-08-15T13:27:35.160Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 diskcache=d6c1e428-7931-4683-875a-7c2fc4c86e57 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:04.888" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:34.988" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.369891] gitaly[904]: time="2026-08-15T13:27:35.160Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=193 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7440 command.oublock=0 command.real_time_ms=164 command.spawn_token_fork_ms=1 command.system_time_ms=90 command.user_time_ms=103 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:04.888" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:34.988" grpc.time_ms=171.534 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.497951] gitaly[904]: time="2026-08-15T13:27:35.328Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-15T13:28:05.326" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.327" grpc.time_ms=1.387 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.580577] gitaly[904]: time="2026-08-15T13:27:35.411Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKV4M24Z4CKVZNQKQB0B2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:05.409" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.409" grpc.time_ms=1.251 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 427.627401] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.456Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"56c176e6502c945b1b274e32","created_at":"2026-08-15T13:26:09.342Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-1e89151f85f94d2b","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:26:09.350Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-56c176e6502c945b1b274e32: done: 0.68122 sec","job_status":"done","queue_duration_s":85.424807,"scheduling_latency_s":85.424899,"gitaly_calls":3,"gitaly_duration_s":0.185526,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.036685,"redis_read_bytes":457,"redis_write_bytes":27999,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001888,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00265,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.012722,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.004961,"redis_queues_read_bytes":36,"redis_queues_write_bytes":14493,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.007844,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000897,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000749,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.003086,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.094,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.035,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.042,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.045,"db_ci_duration_s":0.049,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.372051,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.57435448,"duration_s":0.68122,"completed_at":"2026-08-15T13:27:35.455Z","load_balancing_strategy":"primary","db_duration_s":0.110608,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.722442] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.460Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d718ec66d6a3a3228fd6a6b6","created_at":"2026-08-15T13:27:32.668Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.699Z","job_size_bytes":46,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d718ec66d6a3a3228fd6a6b6: start","job_status":"start","queue_duration_s":2.760293,"scheduling_latency_s":2.76034} gitlab # [ 427.743336] gitaly[904]: time="2026-08-15T13:27:35.568Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-08-15T19:27:35.555" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:35.556" grpc.time_ms=11.71 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.766976] nixbot[3556]: INFO: ::ffff:127.0.0.1:60914 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 427.770160] gitaly[904]: time="2026-08-15T13:27:35.600Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=12 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-15T13:28:05.583" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.584" grpc.time_ms=16.074 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.828616] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.658Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"d718ec66d6a3a3228fd6a6b6","created_at":"2026-08-15T13:27:32.668Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.699Z","job_size_bytes":46,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-d718ec66d6a3a3228fd6a6b6: done: 0.198471 sec","job_status":"done","queue_duration_s":2.760293,"scheduling_latency_s":2.76034,"gitaly_calls":2,"gitaly_duration_s":0.035604,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.014158,"redis_read_bytes":69,"redis_write_bytes":4884,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.002445,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.001045,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002812,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.006662,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001194,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.127612,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00068018,"duration_s":0.198471,"completed_at":"2026-08-15T13:27:35.658Z","load_balancing_strategy":"primary","db_duration_s":0.019048,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 427.907480] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.661Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"747f0e630605d3b0bb1f7fa3","created_at":"2026-08-15T13:27:32.730Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.740Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-747f0e630605d3b0bb1f7fa3: start","job_status":"start","queue_duration_s":2.920695,"scheduling_latency_s":2.920728} gitlab # [ 427.930746] gitaly[904]: time="2026-08-15T13:27:35.738Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 diskcache=33aee477-7204-4265-b231-16774ce23943 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-15T19:27:35.702" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.703" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.942660] gitaly[904]: time="2026-08-15T13:27:35.738Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=23 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=39 command.spawn_token_fork_ms=2 command.system_time_ms=20 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-15T19:27:35.702" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.703" grpc.time_ms=35.23 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 427.993067] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.822Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"747f0e630605d3b0bb1f7fa3","created_at":"2026-08-15T13:27:32.730Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.740Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-747f0e630605d3b0bb1f7fa3: done: 0.161261 sec","job_status":"done","queue_duration_s":2.920695,"scheduling_latency_s":2.920728,"gitaly_calls":1,"gitaly_duration_s":0.038931,"redis_calls":15,"redis_duration_s":0.0061530000000000005,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000676,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.00096,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001954,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.022,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.043,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.016,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.085687,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.11464266,"duration_s":0.161261,"completed_at":"2026-08-15T13:27:35.822Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 428.056643] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.826Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0629e76e31634a459cb5a4b3","created_at":"2026-08-15T13:27:32.928Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.931Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0629e76e31634a459cb5a4b3: start","job_status":"start","queue_duration_s":2.895161,"scheduling_latency_s":2.895205} gitlab # [ 428.076900] gitaly[904]: time="2026-08-15T13:27:35.907Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=9 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-08-15T13:28:05.890" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.891" grpc.time_ms=16.332 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.097182] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.926Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"0629e76e31634a459cb5a4b3","created_at":"2026-08-15T13:27:32.928Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:32.931Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-0629e76e31634a459cb5a4b3: done: 0.100317 sec","job_status":"done","queue_duration_s":2.895161,"scheduling_latency_s":2.895205,"gitaly_calls":1,"gitaly_duration_s":0.019882,"redis_calls":16,"redis_duration_s":0.007202,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000468,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001918,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.003427,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001389,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.064085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.100317,"completed_at":"2026-08-15T13:27:35.926Z","load_balancing_strategy":"primary","db_duration_s":0.005849,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 428.146563] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:35.929Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b5079f011e6f3bb618cbce90","created_at":"2026-08-15T13:27:33.731Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:991b490951ad546ea8ab694b420982c203427b6e8256e9d69c3e0027432e97dd","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:33.737Z","job_size_bytes":467,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b5079f011e6f3bb618cbce90: start","job_status":"start","queue_duration_s":2.192206,"scheduling_latency_s":2.192245} gitlab # [ 428.164126] gitaly[904]: time="2026-08-15T13:27:35.975Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:05.073" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:35.973" grpc.time_ms=1.158 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.176146] gitaly[904]: time="2026-08-15T13:27:35.995Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=2 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-15T13:27:45.068" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:35.979" grpc.time_ms=16.044 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.386672] gitaly[904]: time="2026-08-15T13:27:36.216Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-15T13:28:06.213" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:36.214" grpc.time_ms=1.578 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.425259] gitaly[904]: time="2026-08-15T13:27:36.227Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=2 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=46 command.oublock=0 command.real_time_ms=3 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-15T13:28:06.221" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:36.222" grpc.time_ms=4.664 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 428.528641] gitaly[904]: time="2026-08-15T13:27:36.359Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=1 command.system_time_ms=13 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:46.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:36.343" grpc.time_ms=16.039 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 428.836008] nixbot[3556]: INFO: ::ffff:127.0.0.1:60920 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab # [ 429.181263] gitaly[904]: time="2026-08-15T13:27:37.011Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=20 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=1 command.system_time_ms=20 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-15T13:28:06.085" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:36.986" grpc.time_ms=25.184 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.224993] gitaly[904]: time="2026-08-15T13:27:37.031Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKMPQJTK5WAAWAJC6G8Q5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-15T13:28:07.017" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.017" grpc.time_ms=13.869 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.266744] sidekiq-cluster[3536]: {"severity":"WARN","time":"2026-08-15T13:27:37.096Z","class":"CreatePipelineWorker","correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"4b2631ed070ad5751ddf024b91e967d30ab33e37","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4b2631ed070ad5751ddf024b91e967d30ab33e37","ref":"refs/heads/master"},"retry":0} gitlab # [ 429.284678] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.114Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"b5079f011e6f3bb618cbce90","created_at":"2026-08-15T13:27:33.731Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:991b490951ad546ea8ab694b420982c203427b6e8256e9d69c3e0027432e97dd","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:33.737Z","job_size_bytes":467,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-b5079f011e6f3bb618cbce90: done: 1.184147 sec","job_status":"done","queue_duration_s":2.192206,"scheduling_latency_s":2.192245,"gitaly_calls":7,"gitaly_duration_s":0.103728,"redis_calls":51,"redis_duration_s":0.016493999999999998,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.001503,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.00697,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002439,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003744,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001838,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.041,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.998091,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000946,"duration_s":1.184147,"completed_at":"2026-08-15T13:27:37.113Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.084507,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 429.370953] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.118Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"b497fc9d-40bc-4326-8cd7-1b366917d1a1\"}"],"class":"WebHookWorker","jid":"dec006279fece2e4ccb60946","created_at":"2026-08-15T13:27:33.882Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:94a8a6788927719dcf5c6f199fa8417e23d42427fbeab1df89ce27d6f25155af","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:33.888Z","job_size_bytes":1751,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dec006279fece2e4ccb60946: start","job_status":"start","queue_duration_s":3.229417,"scheduling_latency_s":3.229455} gitlab # [ 429.512345] nixbot[3556]: INFO: ::ffff:127.0.0.1:60934 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 429.555981] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.385Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"b497fc9d-40bc-4326-8cd7-1b366917d1a1\"}"],"class":"WebHookWorker","jid":"dec006279fece2e4ccb60946","created_at":"2026-08-15T13:27:33.882Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-9db2c0b3e25c4791","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:94a8a6788927719dcf5c6f199fa8417e23d42427fbeab1df89ce27d6f25155af","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:33.888Z","job_size_bytes":1751,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-dec006279fece2e4ccb60946: done: 0.267411 sec","job_status":"done","queue_duration_s":3.229417,"scheduling_latency_s":3.229455,"redis_calls":19,"redis_duration_s":0.008614,"redis_read_bytes":32,"redis_write_bytes":5368,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.003047,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000722,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3730,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00315,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001695,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.09266039999999975,"cpu_s":0.146275,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00099694,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.267411,"completed_at":"2026-08-15T13:27:37.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005857,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.671113] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.388Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"60acb897b204819ea7ee8c9a","created_at":"2026-08-15T13:27:34.040Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"747f0e630605d3b0bb1f7fa3","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.042Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-60acb897b204819ea7ee8c9a: start","job_status":"start","queue_duration_s":3.345658,"scheduling_latency_s":3.345693} gitlab # [ 429.693389] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.472Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"60acb897b204819ea7ee8c9a","created_at":"2026-08-15T13:27:34.040Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"747f0e630605d3b0bb1f7fa3","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.042Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-60acb897b204819ea7ee8c9a: done: 0.083459 sec","job_status":"done","queue_duration_s":3.345658,"scheduling_latency_s":3.345693,"gitaly_calls":1,"gitaly_duration_s":0.01984,"redis_calls":12,"redis_duration_s":0.0046890000000000005,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001799,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001094,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001796,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.012,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.012,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048633,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.04871118,"duration_s":0.083459,"completed_at":"2026-08-15T13:27:37.471Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 429.745754] gitaly[904]: time="2026-08-15T13:27:37.439Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM diskcache=007bc9f1-8653-4a23-ab8a-2efc8dc2b07a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-15T19:27:37.423" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.423" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.756562] gitaly[904]: time="2026-08-15T13:27:37.440Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=291 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-08-15T19:27:37.423" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.423" grpc.time_ms=16.116 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.770020] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:37.474Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2d1c9ffefb5860a4480f557e","created_at":"2026-08-15T13:27:34.198Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5df18c6da2ab4aaefa968273e32570943b09428c16b2d4bd51e4cbf1c274eec2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.201Z","job_size_bytes":467,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2d1c9ffefb5860a4480f557e: start","job_status":"start","queue_duration_s":3.272646,"scheduling_latency_s":3.272674} gitlab # [ 429.787811] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:37 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 192 gitlab # [ 429.790077] gitaly[904]: time="2026-08-15T13:27:37.515Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:07.511" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.511" grpc.time_ms=3.863 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.802985] gitaly[904]: time="2026-08-15T13:27:37.532Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-15T13:27:47.179" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:37.520" grpc.time_ms=12.313 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.816211] gitaly[904]: time="2026-08-15T13:27:37.585Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=8 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SPHNHJECNX9305DPY7WJM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:37.561" grpc.time_ms=23.945 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.829422] gitaly[904]: time="2026-08-15T13:27:37.588Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-15T13:28:07.587" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.587" grpc.time_ms=1.098 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 429.842241] gitaly[904]: time="2026-08-15T13:27:37.597Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=92 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=1 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-08-15T13:28:07.592" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.593" grpc.time_ms=3.46 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.857217] gitaly[904]: time="2026-08-15T13:27:37.639Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=6 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:47.289" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:37.630" grpc.time_ms=9.008 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 429.898286] nixbot[3556]: INFO: ::ffff:127.0.0.1:60946 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.06 seconds) gitlab # [ 429.917695] gitaly[904]: time="2026-08-15T13:27:37.748Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPHVNNF0V69MMD1X5JQ35 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:37.735" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 429.932074] gitaly[904]: time="2026-08-15T13:27:37.748Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPHVNNF0V69MMD1X5JQ35 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:37.735" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.948172] gitlab-workhorse[820]: time="2026-08-15T13:27:37Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 429.950204] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:37 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 160 gitlab # [ 429.952239] gitaly[904]: time="2026-08-15T13:27:37.748Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=2 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPHVNNF0V69MMD1X5JQ35 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:37.735" grpc.time_ms=12.794 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 429.967323] nixbot[3556]: {"ts": "2026-08-15T13:27:37.795Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"} gitlab # [ 430.072229] gitaly[904]: time="2026-08-15T13:27:37.902Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:07.800" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:37.901" grpc.time_ms=1.176 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.269420] gitaly[904]: time="2026-08-15T13:27:38.100Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:48.035" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:38.085" grpc.time_ms=14.224 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.352085] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 370 gitlab # [ 430.355239] nixbot[3556]: {"ts": "2026-08-15T13:27:38.184Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 430.472665] gitaly[904]: time="2026-08-15T13:27:38.303Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJEHDSSS44HQPK94RTVP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:08.100" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:38.301" grpc.time_ms=1.445 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.491113] gitaly[904]: time="2026-08-15T13:27:38.321Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=19 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=880 command.oublock=0 command.real_time_ms=21 command.spawn_token_fork_ms=3 command.system_time_ms=19 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-08-15T13:28:08.295" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:38.295" grpc.time_ms=26.032 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.514398] gitaly[904]: time="2026-08-15T13:27:38.345Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SKTT07FY64FC53972H4DM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-08-15T13:28:08.326" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:38.327" grpc.time_ms=17.174 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.543408] sidekiq-cluster[3536]: {"severity":"WARN","time":"2026-08-15T13:27:38.373Z","class":"CreatePipelineWorker","correlation_id":"01M02SKTT07FY64FC53972H4DM","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"4b2631ed070ad5751ddf024b91e967d30ab33e37","after":"b0d1904a3636a352e89f8292bcc2601f40d8477e","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"b0d1904a3636a352e89f8292bcc2601f40d8477e","ref":"refs/heads/master"},"retry":0} gitlab # [ 430.555984] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.385Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"2d1c9ffefb5860a4480f557e","created_at":"2026-08-15T13:27:34.198Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5df18c6da2ab4aaefa968273e32570943b09428c16b2d4bd51e4cbf1c274eec2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.201Z","job_size_bytes":467,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-2d1c9ffefb5860a4480f557e: done: 0.911004 sec","job_status":"done","queue_duration_s":3.272646,"scheduling_latency_s":3.272674,"gitaly_calls":7,"gitaly_duration_s":0.098338,"redis_calls":33,"redis_duration_s":0.00891,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.001954,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00185,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.003755,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001351,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.780692,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.911004,"completed_at":"2026-08-15T13:27:38.385Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.034042,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.605216] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.389Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"e3c7e4e3-8e54-429d-9cd7-0245b79a2eff\"}"],"class":"WebHookWorker","jid":"82dec683d24e82f33cc5795d","created_at":"2026-08-15T13:27:34.219Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d45748020d37d20ba440190f0e41b3e66141aec7cb9dec2b0b2ed31405bafe68","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.222Z","job_size_bytes":1749,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-82dec683d24e82f33cc5795d: start","job_status":"start","queue_duration_s":4.167015,"scheduling_latency_s":4.167058} gitlab # [ 430.622861] gitaly[904]: time="2026-08-15T13:27:38.430Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJEHDSSS44HQPK94RTVP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:48.176" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:38.417" grpc.time_ms=13.63 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.636572] nixbot[3556]: INFO: ::ffff:127.0.0.1:60950 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted gitlab # [ 430.637874] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.466Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"e3c7e4e3-8e54-429d-9cd7-0245b79a2eff\"}"],"class":"WebHookWorker","jid":"82dec683d24e82f33cc5795d","created_at":"2026-08-15T13:27:34.219Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-49d39f8746244971","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d45748020d37d20ba440190f0e41b3e66141aec7cb9dec2b0b2ed31405bafe68","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.222Z","job_size_bytes":1749,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-82dec683d24e82f33cc5795d: done: 0.077194 sec","job_status":"done","queue_duration_s":4.167015,"scheduling_latency_s":4.167058,"redis_calls":14,"redis_duration_s":0.0046890000000000005,"redis_read_bytes":26,"redis_write_bytes":4683,"redis_queues_calls":2,"redis_queues_duration_s":0.000673,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3727,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002617,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001399,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.012221400000044014,"cpu_s":0.049697,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.077194,"completed_at":"2026-08-15T13:27:38.466Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003331,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.685587] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.470Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a0e1bc0f28b82ffbc45fa386","created_at":"2026-08-15T13:27:34.964Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.972Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a0e1bc0f28b82ffbc45fa386: start","job_status":"start","queue_duration_s":3.497756,"scheduling_latency_s":3.497795} gitlab # [ 430.702224] nixbot[3556]: {"ts": "2026-08-15T13:27:38.506Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 430.704962] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 312 gitlab # [ 430.749329] gitaly[904]: time="2026-08-15T13:27:38.578Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJQFQF93G7MEJX5D4TN6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.568" grpc.time_ms=10.028 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.769894] gitaly[904]: time="2026-08-15T13:27:38.598Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJRJ9SA21WDFEPBYAZ1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:08.397" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:38.598" grpc.time_ms=0.701 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 430.786265] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 99 gitlab # [ 430.805614] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.635Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"a0e1bc0f28b82ffbc45fa386","created_at":"2026-08-15T13:27:34.964Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:34.972Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-a0e1bc0f28b82ffbc45fa386: done: 0.164941 sec","job_status":"done","queue_duration_s":3.497756,"scheduling_latency_s":3.497795,"redis_calls":10,"redis_duration_s":0.006062,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000917,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00377,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001375,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.045,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.009,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.038,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.091449,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00063876,"duration_s":0.164941,"completed_at":"2026-08-15T13:27:38.635Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.056567,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 430.861121] gitaly[904]: time="2026-08-15T13:27:38.640Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJTN67Y552ZQYM8D8RDQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.631" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 430.871766] gitlab-workhorse[820]: time="2026-08-15T13:27:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 430.873824] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 61 gitlab # [ 430.875673] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.638Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2d2141bd222b9ccde7fd0811","created_at":"2026-08-15T13:27:35.168Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.178Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2d2141bd222b9ccde7fd0811: start","job_status":"start","queue_duration_s":3.459787,"scheduling_latency_s":3.45982} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 430.892593] gitaly[904]: time="2026-08-15T13:27:38.640Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJTN67Y552ZQYM8D8RDQ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.631" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.905726] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.707Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"2d2141bd222b9ccde7fd0811","created_at":"2026-08-15T13:27:35.168Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.178Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-2d2141bd222b9ccde7fd0811: done: 0.069593 sec","job_status":"done","queue_duration_s":3.459787,"scheduling_latency_s":3.45982,"redis_calls":10,"redis_duration_s":0.0030480000000000004,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000624,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00205,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.016,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.014,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031636,"duration_s":0.069593,"completed_at":"2026-08-15T13:27:38.707Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015971,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 430.950650] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.709Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"630965022123b6dd6fe8e1ea","created_at":"2026-08-15T13:27:35.198Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.210Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-630965022123b6dd6fe8e1ea: start","job_status":"start","queue_duration_s":3.499646,"scheduling_latency_s":3.499667} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.07 seconds) gitlab # [ 430.966684] gitaly[904]: time="2026-08-15T13:27:38.640Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJTN67Y552ZQYM8D8RDQ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.631" grpc.time_ms=8.358 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 430.987113] nixbot[3556]: INFO: ::ffff:127.0.0.1:60276 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab # [ 430.989260] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.775Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"630965022123b6dd6fe8e1ea","created_at":"2026-08-15T13:27:35.198Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.210Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-630965022123b6dd6fe8e1ea: done: 0.065559 sec","job_status":"done","queue_duration_s":3.499646,"scheduling_latency_s":3.499667,"redis_calls":10,"redis_duration_s":0.003045,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000357,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002252,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000436,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03099,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031134,"duration_s":0.065559,"completed_at":"2026-08-15T13:27:38.775Z","load_balancing_strategy":"primary","db_duration_s":0.019914,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.033306] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.779Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bee8ea3248255ecbf13ee5b8","created_at":"2026-08-15T13:27:35.233Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.241Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bee8ea3248255ecbf13ee5b8: start","job_status":"start","queue_duration_s":3.537928,"scheduling_latency_s":3.53795} gitlab # [ 431.048913] gitaly[904]: time="2026-08-15T13:27:38.812Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=8 command.system_time_ms=0 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJRJ9SA21WDFEPBYAZ1X grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:48.127" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:38.797" grpc.time_ms=14.313 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.063274] nix-daemon[3765]: accepted connection from pid 4326, user nixbot gitlab # [ 431.064315] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 135 gitlab # [ 431.066357] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 365 gitlab # [ 431.068929] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.846Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"bee8ea3248255ecbf13ee5b8","created_at":"2026-08-15T13:27:35.233Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.241Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-bee8ea3248255ecbf13ee5b8: done: 0.066573 sec","job_status":"done","queue_duration_s":3.537928,"scheduling_latency_s":3.53795,"redis_calls":13,"redis_duration_s":0.01274,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001579,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006327,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002404,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00243,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032004,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.001102,"duration_s":0.066573,"completed_at":"2026-08-15T13:27:38.846Z","load_balancing_strategy":"primary","db_duration_s":0.00849,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.113903] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.848Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"47b1e16b638c1cb798e21102","created_at":"2026-08-15T13:27:35.242Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.250Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-47b1e16b638c1cb798e21102: start","job_status":"start","queue_duration_s":3.598499,"scheduling_latency_s":3.59852} gitlab # [ 431.130409] nixbot[3556]: {"ts": "2026-08-15T13:27:38.881Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4b2631ed070ad5751ddf024b91e967d30ab33e37 \"HTTP/1.1 201 Created\""} gitlab # [ 431.134425] gitaly[904]: time="2026-08-15T13:27:38.828Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=4 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJY5T3HZ21XY5XYF73SE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=194 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.812" grpc.time_ms=15.704 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.148114] gitaly[904]: time="2026-08-15T13:27:38.899Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK2E8JJFYPBX4KA1YFH6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.891" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 431.158292] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.889Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"47b1e16b638c1cb798e21102","created_at":"2026-08-15T13:27:35.242Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.250Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-47b1e16b638c1cb798e21102: done: 0.041063 sec","job_status":"done","queue_duration_s":3.598499,"scheduling_latency_s":3.59852,"redis_calls":10,"redis_duration_s":0.005027,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001742,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002932,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040718,"duration_s":0.041063,"completed_at":"2026-08-15T13:27:38.889Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004515,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.202761] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.892Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cb0238642a30312a5bd5cf71","created_at":"2026-08-15T13:27:35.251Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.259Z","job_size_bytes":19,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cb0238642a30312a5bd5cf71: start","job_status":"start","queue_duration_s":3.633179,"scheduling_latency_s":3.6332} gitlab # [ 431.219169] gitlab-workhorse[820]: time="2026-08-15T13:27:38Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 431.221027] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 69 gitlab # [ 431.222828] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:38 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 63 gitlab # [ 431.224903] gitaly[904]: time="2026-08-15T13:27:38.899Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK2E8JJFYPBX4KA1YFH6 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.891" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.238360] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.928Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"cb0238642a30312a5bd5cf71","created_at":"2026-08-15T13:27:35.251Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.259Z","job_size_bytes":19,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-cb0238642a30312a5bd5cf71: done: 0.03625 sec","job_status":"done","queue_duration_s":3.633179,"scheduling_latency_s":3.6332,"redis_calls":11,"redis_duration_s":0.002958,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00047,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00214,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000348,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022219,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022682,"duration_s":0.03625,"completed_at":"2026-08-15T13:27:38.928Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00469,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.283241] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.931Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8101533541cfc88b1377e2d7","created_at":"2026-08-15T13:27:35.263Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.269Z","job_size_bytes":66,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8101533541cfc88b1377e2d7: start","job_status":"start","queue_duration_s":3.662614,"scheduling_latency_s":3.662635} gitlab # [ 431.300907] gitaly[904]: time="2026-08-15T13:27:38.899Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK2E8JJFYPBX4KA1YFH6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.891" grpc.time_ms=8.487 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.315606] gitlab-workhorse[820]: time="2026-08-15T13:27:39Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=216 gitlab # [ 431.317559] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:39 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 100 gitlab # [ 431.319402] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.974Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"8101533541cfc88b1377e2d7","created_at":"2026-08-15T13:27:35.263Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.269Z","job_size_bytes":66,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-8101533541cfc88b1377e2d7: done: 0.042861 sec","job_status":"done","queue_duration_s":3.662614,"scheduling_latency_s":3.662635,"redis_calls":10,"redis_duration_s":0.009315,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000328,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00316,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005827,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002458,"duration_s":0.042861,"completed_at":"2026-08-15T13:27:38.974Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004869,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.366373] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:38.976Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9b0bc5ee652a93d14ec7a2ee","created_at":"2026-08-15T13:27:35.270Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.275Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9b0bc5ee652a93d14ec7a2ee: start","job_status":"start","queue_duration_s":3.700065,"scheduling_latency_s":3.700089} gitlab # [ 431.383120] gitaly[904]: time="2026-08-15T13:27:38.998Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=6 command.system_time_ms=2 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SPK5QETZ2XAYK6MA39PCX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:38.990" grpc.time_ms=7.635 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.396548] gitaly[904]: time="2026-08-15T13:27:39.100Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK7SSFMWH2ZH785PR8QS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:39.074" pid=904 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc gitlab # [ 431.406710] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.019Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"9b0bc5ee652a93d14ec7a2ee","created_at":"2026-08-15T13:27:35.270Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.275Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-9b0bc5ee652a93d14ec7a2ee: done: 0.043333 sec","job_status":"done","queue_duration_s":3.700065,"scheduling_latency_s":3.700089,"redis_calls":10,"redis_duration_s":0.001442,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000271,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00085,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000321,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00022898,"duration_s":0.043333,"completed_at":"2026-08-15T13:27:39.019Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006094,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.451320] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.020Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"177a6834b82650d55e362e50","created_at":"2026-08-15T13:27:35.284Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.290Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-177a6834b82650d55e362e50: start","job_status":"start","queue_duration_s":3.730377,"scheduling_latency_s":3.730396} gitlab # [ 431.468032] gitaly[904]: time="2026-08-15T13:27:39.100Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK7SSFMWH2ZH785PR8QS grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:39.074" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.481819] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.057Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"177a6834b82650d55e362e50","created_at":"2026-08-15T13:27:35.284Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.290Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-177a6834b82650d55e362e50: done: 0.037047 sec","job_status":"done","queue_duration_s":3.730377,"scheduling_latency_s":3.730396,"redis_calls":10,"redis_duration_s":0.001313,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000261,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000771,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024878,"duration_s":0.037047,"completed_at":"2026-08-15T13:27:39.057Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00846,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.526964] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2d687ce44565106a5c005820","created_at":"2026-08-15T13:27:35.290Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.295Z","job_size_bytes":6,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2d687ce44565106a5c005820: start","job_status":"start","queue_duration_s":3.763093,"scheduling_latency_s":3.76311} gitlab # [ 431.544084] gitaly[904]: time="2026-08-15T13:27:39.100Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=22 command.spawn_token_fork_ms=3 command.system_time_ms=18 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPK7SSFMWH2ZH785PR8QS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:39.074" grpc.time_ms=25.825 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 431.559395] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.083Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"2d687ce44565106a5c005820","created_at":"2026-08-15T13:27:35.290Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.295Z","job_size_bytes":6,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-2d687ce44565106a5c005820: done: 0.024603 sec","job_status":"done","queue_duration_s":3.763093,"scheduling_latency_s":3.76311,"redis_calls":10,"redis_duration_s":0.001959,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00034,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000275,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014552,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025378,"duration_s":0.024603,"completed_at":"2026-08-15T13:27:39.083Z","load_balancing_strategy":"primary","db_duration_s":0.003242,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.603120] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.084Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"190e9522f260538906d80faf","created_at":"2026-08-15T13:27:35.296Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.301Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-190e9522f260538906d80faf: start","job_status":"start","queue_duration_s":3.782934,"scheduling_latency_s":3.78295} gitlab # [ 431.618714] gitaly[904]: time="2026-08-15T13:27:39.194Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:09.092" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:39.192" grpc.time_ms=0.898 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.631590] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.133Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"190e9522f260538906d80faf","created_at":"2026-08-15T13:27:35.296Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.301Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-190e9522f260538906d80faf: done: 0.048528 sec","job_status":"done","queue_duration_s":3.782934,"scheduling_latency_s":3.78295,"redis_calls":10,"redis_duration_s":0.001769,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000295,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001094,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00038,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002931,"duration_s":0.048528,"completed_at":"2026-08-15T13:27:39.133Z","load_balancing_strategy":"primary","db_duration_s":0.012618,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.674835] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.134Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a9deb55f9ae7491f909c7704","created_at":"2026-08-15T13:27:35.308Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.313Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a9deb55f9ae7491f909c7704: start","job_status":"start","queue_duration_s":3.820641,"scheduling_latency_s":3.820661} gitlab # [ 431.691740] gitaly[904]: time="2026-08-15T13:27:39.224Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=590 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-08-15T13:28:09.106" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:39.206" grpc.time_ms=17.378 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.706041] nixbot[3556]: {"ts": "2026-08-15T13:27:39.469Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e \"HTTP/1.1 201 Created\""} gitlab # [ 431.708785] nixbot[3556]: {"ts": "2026-08-15T13:27:39.476Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-b0d1904a3636-4"} gitlab # [ 431.711314] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.213Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a9deb55f9ae7491f909c7704","created_at":"2026-08-15T13:27:35.308Z","trace_propagation_headers":{"sentry-trace":"b09a2d7e310546d99462f47f0d565758-2bf189a8d8fa46ce","baggage":"sentry-trace_id=b09a2d7e310546d99462f47f0d565758,sentry-release=Unknown"},"correlation_id":"01M02SKV4M24Z4CKVZNQKQB0B2","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.313Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a9deb55f9ae7491f909c7704: done: 0.07855 sec","job_status":"done","queue_duration_s":3.820641,"scheduling_latency_s":3.820661,"redis_calls":14,"redis_duration_s":0.004246,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000537,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00163,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046893,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039178,"duration_s":0.07855,"completed_at":"2026-08-15T13:27:39.212Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021709,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.756901] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"7b9c66905e22fb1ab57d9c72","created_at":"2026-08-15T13:27:35.620Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-94c9d01ed7154de4","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.631Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-7b9c66905e22fb1ab57d9c72: start","job_status":"start","queue_duration_s":3.582773,"scheduling_latency_s":3.58279} gitlab # [ 431.772555] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:39 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 201 631 "" "python-httpx/0.28.1" 325 gitlab # [ 431.775347] gitaly[904]: time="2026-08-15T13:27:39.240Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:49.039" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:39.230" grpc.time_ms=10.815 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.789039] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"7b9c66905e22fb1ab57d9c72","created_at":"2026-08-15T13:27:35.620Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-94c9d01ed7154de4","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:35.631Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-7b9c66905e22fb1ab57d9c72: done: 0.066101 sec","job_status":"done","queue_duration_s":3.582773,"scheduling_latency_s":3.58279,"redis_calls":19,"redis_duration_s":0.010102,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001858,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000471,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.004018,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.003755,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.005,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.038117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.011373139999999999,"duration_s":0.066101,"completed_at":"2026-08-15T13:27:39.280Z","load_balancing_strategy":"primary","db_duration_s":0.015046,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.834409] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.282Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dec006279fece2e4ccb60946"],"class":"WebHooks::LogExecutionWorker","jid":"fb5963f1fa16984f4945241e","created_at":"2026-08-15T13:27:37.348Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-b773a14cf9f940be","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97b58f8425057662aeac4de0dd96c0c87f52c2c19e07f584b9d016233b747529","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:37.363Z","job_size_bytes":2593,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fb5963f1fa16984f4945241e: start","job_status":"start","queue_duration_s":1.918502,"scheduling_latency_s":1.918523} gitlab # [ 431.851849] nix-daemon[3765]: accepted connection from pid 4406, user nixbot gitlab # [ 431.852928] nix-daemon[3765]: accepted connection from pid 4424, user nixbot gitlab # [ 431.853877] gitaly[904]: time="2026-08-15T13:27:39.259Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-08-15T13:27:49.069" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:39.259" grpc.time_ms=0.23 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.864939] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.337Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","dec006279fece2e4ccb60946"],"class":"WebHooks::LogExecutionWorker","jid":"fb5963f1fa16984f4945241e","created_at":"2026-08-15T13:27:37.348Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-b773a14cf9f940be","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:97b58f8425057662aeac4de0dd96c0c87f52c2c19e07f584b9d016233b747529","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:37.363Z","job_size_bytes":2593,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-fb5963f1fa16984f4945241e: done: 0.055041 sec","job_status":"done","queue_duration_s":1.918502,"scheduling_latency_s":1.918523,"redis_calls":10,"redis_duration_s":0.0014680000000000001,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000826,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000331,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.011,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.011,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033247,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025524,"duration_s":0.055041,"completed_at":"2026-08-15T13:27:39.336Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015832,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 431.910049] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.338Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c644d87cdd94aaca9cd91e29","created_at":"2026-08-15T13:27:38.155Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-41c2b8b09b834351","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:38.162Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c644d87cdd94aaca9cd91e29: start","job_status":"start","queue_duration_s":1.176237,"scheduling_latency_s":1.176258} gitlab # [ 431.926080] nixbot[3556]: {"ts": "2026-08-15T13:27:39.653Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-b0d1904a3636-4", "argv0": "bwrap"} gitlab # [ 431.928453] gitaly[904]: time="2026-08-15T13:27:39.273Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-08-15T13:27:49.074" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:39.264" grpc.time_ms=8.355 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 431.941715] gitaly[904]: time="2026-08-15T13:27:39.308Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:39.307" grpc.time_ms=0.868 pid=904 span.kind=server system=grpc gitlab # [ 431.952781] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.660Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c644d87cdd94aaca9cd91e29","created_at":"2026-08-15T13:27:38.155Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-41c2b8b09b834351","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:38.162Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3536,"job_status":"rescheduled","message":"PipelineProcessWorker JID-c644d87cdd94aaca9cd91e29: rescheduled"} gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e' gitlab # [ 431.967817] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.668Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"c644d87cdd94aaca9cd91e29","created_at":"2026-08-15T13:27:38.155Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-41c2b8b09b834351","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:38.162Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-c644d87cdd94aaca9cd91e29: done: 0.329927 sec","job_status":"done","queue_duration_s":1.176237,"scheduling_latency_s":1.176258,"gitaly_calls":3,"gitaly_duration_s":0.114765,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.016774999999999998,"redis_read_bytes":151,"redis_write_bytes":18015,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000504,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.000997,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.00613,"redis_queues_read_bytes":29,"redis_queues_write_bytes":12314,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005703,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000252,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000405,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00228,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.054,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.008,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.008,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.028,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.153772,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.28063034,"duration_s":0.329927,"completed_at":"2026-08-15T13:27:39.668Z","load_balancing_strategy":"primary","db_duration_s":0.055405,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.017132] gitaly[904]: time="2026-08-15T13:27:39.335Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:39.334" grpc.time_ms=0.347 pid=904 span.kind=server system=grpc gitlab # [ 432.027004] gitaly[904]: time="2026-08-15T13:27:39.344Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS diskcache=3b0b6780-c4dd-4ba4-b806-4c03c64f7019 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:27:49.086" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:39.276" pid=904 ref=refs/keep-around/b0d1904a3636a352e89f8292bcc2601f40d8477e remote_ip="2001:db8:1::1" revision=b0d1904a3636a352e89f8292bcc2601f40d8477e span.kind=server system=grpc username=root gitlab # [ 432.038736] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.670Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","82dec683d24e82f33cc5795d"],"class":"WebHooks::LogExecutionWorker","jid":"e44db303ed526012818b2e0f","created_at":"2026-08-15T13:27:38.444Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-6810537675224468","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:237de871d070b6ccac8eb264bdc867ce921ebddbe3418520992751ca2203ac40","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:38.450Z","job_size_bytes":2592,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e44db303ed526012818b2e0f: start","job_status":"start","queue_duration_s":1.219705,"scheduling_latency_s":1.21973} gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e', in 0.09 seconds) (finished: subtest: push delivers the webhook, in 91.76 seconds) subtest: eval and build post commit statuses gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 432.055587] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.698Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","82dec683d24e82f33cc5795d"],"class":"WebHooks::LogExecutionWorker","jid":"e44db303ed526012818b2e0f","created_at":"2026-08-15T13:27:38.444Z","trace_propagation_headers":{"sentry-trace":"04f36631aa8d4ce4b2eadf5e9ac9993f-6810537675224468","baggage":"sentry-trace_id=04f36631aa8d4ce4b2eadf5e9ac9993f,sentry-release=Unknown"},"correlation_id":"01M02SKTT07FY64FC53972H4DM","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:237de871d070b6ccac8eb264bdc867ce921ebddbe3418520992751ca2203ac40","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:38.450Z","job_size_bytes":2592,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-e44db303ed526012818b2e0f: done: 0.028073 sec","job_status":"done","queue_duration_s":1.219705,"scheduling_latency_s":1.21973,"redis_calls":7,"redis_duration_s":0.001392,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000394,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018743,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028073,"completed_at":"2026-08-15T13:27:39.697Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.099110] gitaly[904]: time="2026-08-15T13:27:39.344Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=60 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5025 command.oublock=8 command.real_time_ms=61 command.spawn_token_fork_ms=2 command.system_time_ms=37 command.user_time_ms=23 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-08-15T13:27:49.086" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-08-15T13:27:39.276" grpc.time_ms=67.94 pid=904 ref=refs/keep-around/b0d1904a3636a352e89f8292bcc2601f40d8477e remote_ip="2001:db8:1::1" revision=b0d1904a3636a352e89f8292bcc2601f40d8477e span.kind=server system=grpc username=root gitlab # [ 432.114624] nixbot[3556]: INFO: ::ffff:127.0.0.1:60286 - "GET /api/repos/gitlab/root/test-flake/builds?commit=b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 200 OK gitlab # [ 432.116620] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.699Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"1018dc4fd3a252b2d231734f","created_at":"2026-08-15T13:27:39.261Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-46362298505744c5","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.269Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-1018dc4fd3a252b2d231734f: start","job_status":"start","queue_duration_s":0.430466,"scheduling_latency_s":0.430485} gitlab # [ 432.133774] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.800Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"1018dc4fd3a252b2d231734f","created_at":"2026-08-15T13:27:39.261Z","trace_propagation_headers":{"sentry-trace":"94c7b8d739794fd5a9f014ee201db694-46362298505744c5","baggage":"sentry-trace_id=94c7b8d739794fd5a9f014ee201db694,sentry-release=Unknown"},"correlation_id":"01M02SKMPQJTK5WAAWAJC6G8Q5","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.269Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-1018dc4fd3a252b2d231734f: done: 0.100855 sec","job_status":"done","queue_duration_s":0.430466,"scheduling_latency_s":0.430485,"redis_calls":11,"redis_duration_s":0.004249,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000449,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.003286,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000514,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.06548,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00039268,"duration_s":0.100855,"completed_at":"2026-08-15T13:27:39.800Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.030155,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.180183] gitaly[904]: time="2026-08-15T13:27:39.349Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:09.248" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:39.349" grpc.time_ms=0.428 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.192591] gitaly[904]: time="2026-08-15T13:27:39.455Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:39.455" grpc.time_ms=0.392 pid=904 span.kind=server system=grpc gitlab # [ 432.202426] nix-daemon[3765]: accepted connection from pid 4435, user nixbot gitlab # [ 432.203407] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.807Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5bbfb713df7d34c4effd487b","created_at":"2026-08-15T13:27:39.352Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.360Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5bbfb713df7d34c4effd487b: start","job_status":"start","queue_duration_s":0.446647,"scheduling_latency_s":0.446666} gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.19 seconds) gitlab # [ 432.220807] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.843Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"5bbfb713df7d34c4effd487b","created_at":"2026-08-15T13:27:39.352Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.360Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-5bbfb713df7d34c4effd487b: done: 0.036193 sec","job_status":"done","queue_duration_s":0.446647,"scheduling_latency_s":0.446666,"redis_calls":7,"redis_duration_s":0.002196,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00153,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000666,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022487,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036193,"completed_at":"2026-08-15T13:27:39.843Z","load_balancing_strategy":"primary","db_duration_s":0.00866,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.263943] gitaly[904]: time="2026-08-15T13:27:39.489Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:39.488" grpc.time_ms=0.471 pid=904 span.kind=server system=grpc gitlab # [ 432.273956] gitaly[904]: time="2026-08-15T13:27:39.514Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:39.514" grpc.time_ms=0.782 pid=904 span.kind=server system=grpc gitlab # [ 432.284056] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:40 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 132 gitlab # [ 432.286626] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.845Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"454710bc186a19c27f443e36","created_at":"2026-08-15T13:27:39.361Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.367Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-454710bc186a19c27f443e36: start","job_status":"start","queue_duration_s":0.478853,"scheduling_latency_s":0.478873} gitlab # [ 432.304169] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.965Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"454710bc186a19c27f443e36","created_at":"2026-08-15T13:27:39.361Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.367Z","job_size_bytes":65,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-454710bc186a19c27f443e36: done: 0.119511 sec","job_status":"done","queue_duration_s":0.478853,"scheduling_latency_s":0.478873,"redis_calls":10,"redis_duration_s":0.008722,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.001934,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.048,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.029,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.056142,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119511,"completed_at":"2026-08-15T13:27:39.965Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.049358,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.347865] nix-daemon[3765]: accepted connection from pid 4454, user nixbot gitlab # [ 432.348875] nix-daemon[3765]: accepted connection from pid 4435, user nixbot gitlab # [ 432.349805] gitaly[904]: time="2026-08-15T13:27:39.521Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 diskcache=a6e7d287-6ac2-4a0f-8c8f-71380704b626 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:09.415" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:39.416" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.360913] gitaly[904]: time="2026-08-15T13:27:39.521Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=96 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7422 command.oublock=0 command.real_time_ms=101 command.spawn_token_fork_ms=0 command.system_time_ms=50 command.user_time_ms=46 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:09.415" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:39.416" grpc.time_ms=105.408 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.374041] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:39.967Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9e939fb8eb5363f69e0d695c","created_at":"2026-08-15T13:27:39.406Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.409Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9e939fb8eb5363f69e0d695c: start","job_status":"start","queue_duration_s":0.558856,"scheduling_latency_s":0.558875} gitlab # [ 432.390575] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.014Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"9e939fb8eb5363f69e0d695c","created_at":"2026-08-15T13:27:39.406Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.409Z","job_size_bytes":23,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-9e939fb8eb5363f69e0d695c: done: 0.046617 sec","job_status":"done","queue_duration_s":0.558856,"scheduling_latency_s":0.558875,"redis_calls":7,"redis_duration_s":0.003565,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000872,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002693,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033045,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046617,"completed_at":"2026-08-15T13:27:40.014Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008087,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.433133] gitaly[904]: time="2026-08-15T13:27:39.598Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-15T13:28:09.596" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:39.597" grpc.time_ms=0.738 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.445879] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.015Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f95f7f45ecb228d2fb9c49cc","created_at":"2026-08-15T13:27:39.445Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.448Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f95f7f45ecb228d2fb9c49cc: start","job_status":"start","queue_duration_s":0.566782,"scheduling_latency_s":0.5668} gitlab # [ 432.461537] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.147Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"f95f7f45ecb228d2fb9c49cc","created_at":"2026-08-15T13:27:39.445Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-1b7e12f39dec4a33","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.448Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-f95f7f45ecb228d2fb9c49cc: done: 0.131656 sec","job_status":"done","queue_duration_s":0.566782,"scheduling_latency_s":0.5668,"gitaly_calls":2,"gitaly_duration_s":0.007507,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.006436,"redis_read_bytes":80,"redis_write_bytes":5122,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000831,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001243,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.00024,"redis_queues_read_bytes":3,"redis_queues_write_bytes":1133,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001318,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.00018,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000465,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001328,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.021,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.002,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.004,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102207,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.09982388,"duration_s":0.131656,"completed_at":"2026-08-15T13:27:40.147Z","load_balancing_strategy":"primary","db_duration_s":0.022715,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 432.512637] gitaly[904]: time="2026-08-15T13:27:39.642Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPJ2KEAH04E9DGEKMDDZ8 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:09.641" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:39.641" grpc.time_ms=0.678 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.524919] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c2b1d6138ba64d315823499f","created_at":"2026-08-15T13:27:39.527Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.530Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c2b1d6138ba64d315823499f: start","job_status":"start","queue_duration_s":0.618327,"scheduling_latency_s":0.618345} gitlab # [ 432.541631] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.178Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"c2b1d6138ba64d315823499f","created_at":"2026-08-15T13:27:39.527Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.530Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-c2b1d6138ba64d315823499f: done: 0.029927 sec","job_status":"done","queue_duration_s":0.618327,"scheduling_latency_s":0.618345,"redis_calls":7,"redis_duration_s":0.001605,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001195,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00041,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019159,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029927,"completed_at":"2026-08-15T13:27:40.178Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006466,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.585541] gitaly[904]: time="2026-08-15T13:27:40.031Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPM55H4KSZS4KV6EEGAGZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:10.030" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:40.030" grpc.time_ms=0.666 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.598818] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.180Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6d0f7093fe3f037a067bb418","created_at":"2026-08-15T13:27:39.540Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.542Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6d0f7093fe3f037a067bb418: start","job_status":"start","queue_duration_s":0.637497,"scheduling_latency_s":0.637515} gitlab # [ 432.614679] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.214Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"6d0f7093fe3f037a067bb418","created_at":"2026-08-15T13:27:39.540Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.542Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-6d0f7093fe3f037a067bb418: done: 0.033977 sec","job_status":"done","queue_duration_s":0.637497,"scheduling_latency_s":0.637515,"redis_calls":7,"redis_duration_s":0.001966,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000596,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020583,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033977,"completed_at":"2026-08-15T13:27:40.214Z","load_balancing_strategy":"primary","db_duration_s":0.009154,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.656360] gitaly[904]: time="2026-08-15T13:27:40.091Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-08-15T13:28:10.090" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:40.090" grpc.time_ms=0.893 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.668894] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.215Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c234a59a1dac6d176787e3bf","created_at":"2026-08-15T13:27:39.552Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.555Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c234a59a1dac6d176787e3bf: start","job_status":"start","queue_duration_s":0.660475,"scheduling_latency_s":0.660492} gitlab # [ 432.684704] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.249Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"c234a59a1dac6d176787e3bf","created_at":"2026-08-15T13:27:39.552Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.555Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-c234a59a1dac6d176787e3bf: done: 0.033976 sec","job_status":"done","queue_duration_s":0.660475,"scheduling_latency_s":0.660492,"redis_calls":10,"redis_duration_s":0.002441,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001269,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000694,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000478,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021199,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033976,"completed_at":"2026-08-15T13:27:40.249Z","load_balancing_strategy":"primary","db_duration_s":0.008194,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.728298] gitaly[904]: time="2026-08-15T13:27:40.135Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPKC58YJZZP1ASTE0H2YS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:10.133" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:40.133" grpc.time_ms=0.991 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 432.740764] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.251Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"933851656786a1a26ff31905","created_at":"2026-08-15T13:27:39.556Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.559Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-933851656786a1a26ff31905: start","job_status":"start","queue_duration_s":0.691381,"scheduling_latency_s":0.691399} gitlab # [ 432.757323] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.280Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"933851656786a1a26ff31905","created_at":"2026-08-15T13:27:39.556Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.559Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-933851656786a1a26ff31905: done: 0.029303 sec","job_status":"done","queue_duration_s":0.691381,"scheduling_latency_s":0.691399,"redis_calls":7,"redis_duration_s":0.001935,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001446,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000489,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018071,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.029303,"completed_at":"2026-08-15T13:27:40.280Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006868,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.802492] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.282Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7f860d9f858b056d2c625663","created_at":"2026-08-15T13:27:39.563Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.566Z","job_size_bytes":19,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7f860d9f858b056d2c625663: start","job_status":"start","queue_duration_s":0.715927,"scheduling_latency_s":0.715946} gitlab # [ 432.847992] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.310Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"7f860d9f858b056d2c625663","created_at":"2026-08-15T13:27:39.563Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.566Z","job_size_bytes":19,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-7f860d9f858b056d2c625663: done: 0.028125 sec","job_status":"done","queue_duration_s":0.715927,"scheduling_latency_s":0.715946,"redis_calls":8,"redis_duration_s":0.0021130000000000003,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001654,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028125,"completed_at":"2026-08-15T13:27:40.310Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007399,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.912301] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.312Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6d30fd73d4ec430023151136","created_at":"2026-08-15T13:27:39.568Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.570Z","job_size_bytes":66,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6d30fd73d4ec430023151136: start","job_status":"start","queue_duration_s":0.741827,"scheduling_latency_s":0.741845} gitlab # [ 432.930760] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.337Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"6d30fd73d4ec430023151136","created_at":"2026-08-15T13:27:39.568Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.570Z","job_size_bytes":66,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-6d30fd73d4ec430023151136: done: 0.025158 sec","job_status":"done","queue_duration_s":0.741827,"scheduling_latency_s":0.741845,"redis_calls":7,"redis_duration_s":0.002083,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001528,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000555,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015845,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025158,"completed_at":"2026-08-15T13:27:40.337Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004445,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 432.974002] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.340Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"81048a966d5b01afec23ab96","created_at":"2026-08-15T13:27:39.571Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.574Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-81048a966d5b01afec23ab96: start","job_status":"start","queue_duration_s":0.766135,"scheduling_latency_s":0.766154} gitlab # [ 432.991306] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.369Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"81048a966d5b01afec23ab96","created_at":"2026-08-15T13:27:39.571Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.574Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-81048a966d5b01afec23ab96: done: 0.028787 sec","job_status":"done","queue_duration_s":0.766135,"scheduling_latency_s":0.766154,"redis_calls":7,"redis_duration_s":0.0018880000000000001,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00136,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000528,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017132,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028787,"completed_at":"2026-08-15T13:27:40.369Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007218,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.033509] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.370Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"5ca51b2dcff5241439942c27","created_at":"2026-08-15T13:27:39.574Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.577Z","job_size_bytes":6,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-5ca51b2dcff5241439942c27: start","job_status":"start","queue_duration_s":0.793562,"scheduling_latency_s":0.793582} gitlab # [ 433.049061] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.386Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"5ca51b2dcff5241439942c27","created_at":"2026-08-15T13:27:39.574Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.577Z","job_size_bytes":6,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-5ca51b2dcff5241439942c27: done: 0.015409 sec","job_status":"done","queue_duration_s":0.793562,"scheduling_latency_s":0.793582,"redis_calls":7,"redis_duration_s":0.002251,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000913,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015409,"completed_at":"2026-08-15T13:27:40.386Z","load_balancing_strategy":"primary","db_duration_s":0.000612,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.090436] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.387Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"b11893841f94e7f6ecdeb5d6","created_at":"2026-08-15T13:27:39.578Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.580Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-b11893841f94e7f6ecdeb5d6: start","job_status":"start","queue_duration_s":0.807667,"scheduling_latency_s":0.807684} gitlab # [ 433.105705] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.413Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"b11893841f94e7f6ecdeb5d6","created_at":"2026-08-15T13:27:39.578Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.580Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-b11893841f94e7f6ecdeb5d6: done: 0.026188 sec","job_status":"done","queue_duration_s":0.807667,"scheduling_latency_s":0.807684,"redis_calls":7,"redis_duration_s":0.0019299999999999999,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00137,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00056,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016065,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026188,"completed_at":"2026-08-15T13:27:40.413Z","load_balancing_strategy":"primary","db_duration_s":0.005607,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.147388] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.415Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9dfef7f0f7a3e6a51d785e4c","created_at":"2026-08-15T13:27:39.660Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.662Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9dfef7f0f7a3e6a51d785e4c: start","job_status":"start","queue_duration_s":0.752846,"scheduling_latency_s":0.752862} gitlab # [ 433.163259] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.444Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"9dfef7f0f7a3e6a51d785e4c","created_at":"2026-08-15T13:27:39.660Z","trace_propagation_headers":{"sentry-trace":"966c1fc6720d45ff80a0d3c47907b712-1120a930cbbb4337","baggage":"sentry-trace_id=966c1fc6720d45ff80a0d3c47907b712,sentry-release=Unknown"},"correlation_id":"01M02SPJ2KEAH04E9DGEKMDDZ8","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:39.662Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-9dfef7f0f7a3e6a51d785e4c: done: 0.028997 sec","job_status":"done","queue_duration_s":0.752846,"scheduling_latency_s":0.752862,"redis_calls":8,"redis_duration_s":0.002205,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001742,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000463,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028997,"completed_at":"2026-08-15T13:27:40.444Z","load_balancing_strategy":"primary","db_duration_s":0.006136,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 433.206503] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.446Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8875557cfb0971eff5dd95dc","created_at":"2026-08-15T13:27:40.074Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-20f13f8bdcd7448f","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:40.076Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8875557cfb0971eff5dd95dc: start","job_status":"start","queue_duration_s":0.369649,"scheduling_latency_s":0.369667} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 433.222571] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:40.515Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"8875557cfb0971eff5dd95dc","created_at":"2026-08-15T13:27:40.074Z","trace_propagation_headers":{"sentry-trace":"e56a3b15719a49c8b457c937c5021152-20f13f8bdcd7448f","baggage":"sentry-trace_id=e56a3b15719a49c8b457c937c5021152,sentry-release=Unknown"},"correlation_id":"01M02SPKC58YJZZP1ASTE0H2YS","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:40.076Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-8875557cfb0971eff5dd95dc: done: 0.069306 sec","job_status":"done","queue_duration_s":0.369649,"scheduling_latency_s":0.369667,"redis_calls":7,"redis_duration_s":0.0021279999999999997,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001563,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.026,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.039625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069306,"completed_at":"2026-08-15T13:27:40.515Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.025683,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 433.448509] gitaly[904]: time="2026-08-15T13:27:41.277Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPNBN7DVHCRGCCEPYP5P6 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:11.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:41.276" grpc.time_ms=1.297 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.25 seconds) gitlab # [ 433.499795] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:41 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 155 gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 434.671333] gitaly[904]: time="2026-08-15T13:27:42.501Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPPHJ5RD8XNDFV603YXAM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:12.298" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:42.499" grpc.time_ms=1.163 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.25 seconds) gitlab # [ 434.747136] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:42 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 190 gitlab # [ 435.133338] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:42.962Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3d2574ebdd1dd26a7697a7dd","created_at":"2026-08-15T13:27:42.935Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"678a73427c8dec0a233a0b94ac3506e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:42.958Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3d2574ebdd1dd26a7697a7dd: start","job_status":"start","queue_duration_s":0.003748,"scheduling_latency_s":0.003809} gitlab # [ 435.454858] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.284Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"3d2574ebdd1dd26a7697a7dd","created_at":"2026-08-15T13:27:42.935Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"678a73427c8dec0a233a0b94ac3506e4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:42.958Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-3d2574ebdd1dd26a7697a7dd: done: 0.322116 sec","job_status":"done","queue_duration_s":0.003748,"scheduling_latency_s":0.003809,"redis_calls":13,"redis_duration_s":0.033002000000000004,"redis_read_bytes":11,"redis_write_bytes":1780,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.005417,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.013946,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.013639,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.102,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.08,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.114352,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00458408,"exclusive_lock_hold_duration_s":0.16121198,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.322116,"completed_at":"2026-08-15T13:27:43.284Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.111706,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.549764] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.297Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b595b805a492f0388d42e253","created_at":"2026-08-15T13:27:42.978Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50aad935e88c618a8e0a7794c92c705b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:42.990Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b595b805a492f0388d42e253: start","job_status":"start","queue_duration_s":0.306761,"scheduling_latency_s":0.306811} gitlab # [ 435.567402] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.395Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0} gitlab # [ 435.601772] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.431Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"b595b805a492f0388d42e253","created_at":"2026-08-15T13:27:42.978Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"50aad935e88c618a8e0a7794c92c705b","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:42.990Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-b595b805a492f0388d42e253: done: 0.134184 sec","job_status":"done","queue_duration_s":0.306761,"scheduling_latency_s":0.306811,"redis_calls":9,"redis_duration_s":0.016871999999999998,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002777,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008844,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005251,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031746,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.134184,"completed_at":"2026-08-15T13:27:43.431Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.012169,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.710004] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.450Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9a0b6684df181990045db870","created_at":"2026-08-15T13:27:42.998Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f25c39321675d978fe03196f3af1090b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.025Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9a0b6684df181990045db870: start","job_status":"start","queue_duration_s":0.424472,"scheduling_latency_s":0.424522} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 435.752383] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.582Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"9a0b6684df181990045db870","created_at":"2026-08-15T13:27:42.998Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f25c39321675d978fe03196f3af1090b","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.025Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-9a0b6684df181990045db870: done: 0.13178 sec","job_status":"done","queue_duration_s":0.424472,"scheduling_latency_s":0.424522,"redis_calls":9,"redis_duration_s":0.01687,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002907,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007355,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006608,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.040362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.13178,"completed_at":"2026-08-15T13:27:43.581Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009512,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 435.819525] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:43.594Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fdcd7311ec5812e8c21c72c6","created_at":"2026-08-15T13:27:43.068Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12cbc35eb3bf2c4af124192be6397908","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.145Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fdcd7311ec5812e8c21c72c6: start","job_status":"start","queue_duration_s":0.449472,"scheduling_latency_s":0.44958} gitlab # [ 435.915720] gitaly[904]: time="2026-08-15T13:27:43.745Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPQSKVSJT1K8G581067ZF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:13.643" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:43.743" grpc.time_ms=1.107 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 435.956424] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:43 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses HTTP/1.1" 200 633 "" "curl/8.21.0" 118 gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.21 seconds) gitlab # [ 436.203795] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.031Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"fdcd7311ec5812e8c21c72c6","created_at":"2026-08-15T13:27:43.068Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"12cbc35eb3bf2c4af124192be6397908","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.145Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-fdcd7311ec5812e8c21c72c6: done: 0.436167 sec","job_status":"done","queue_duration_s":0.449472,"scheduling_latency_s":0.44958,"redis_calls":11,"redis_duration_s":0.056285,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.006501,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005676,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.044108,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.054846,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00335614,"exclusive_lock_hold_duration_s":0.04724516,"duration_s":0.436167,"completed_at":"2026-08-15T13:27:44.030Z","load_balancing_strategy":"primary","db_duration_s":0.043592,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.300882] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.047Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ad353873b85a959d7680b173","created_at":"2026-08-15T13:27:43.208Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8c14e1f79d6ca3e563134db14f0d447","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.233Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ad353873b85a959d7680b173: start","job_status":"start","queue_duration_s":0.814269,"scheduling_latency_s":0.81432} gitlab # [ 436.375183] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.204Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"ad353873b85a959d7680b173","created_at":"2026-08-15T13:27:43.208Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a8c14e1f79d6ca3e563134db14f0d447","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.233Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-ad353873b85a959d7680b173: done: 0.157453 sec","job_status":"done","queue_duration_s":0.814269,"scheduling_latency_s":0.81432,"redis_calls":9,"redis_duration_s":0.012116,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002864,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004788,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004464,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.047469,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.157453,"completed_at":"2026-08-15T13:27:44.204Z","load_balancing_strategy":"primary","db_duration_s":0.030429,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.463305] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.208Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f4541085a4d7750691e1fac4","created_at":"2026-08-15T13:27:43.250Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1541fb1c881ac731cf6f93ebeb9d7a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.276Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f4541085a4d7750691e1fac4: start","job_status":"start","queue_duration_s":0.931588,"scheduling_latency_s":0.931637} gitlab # [ 436.815304] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.644Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"f4541085a4d7750691e1fac4","created_at":"2026-08-15T13:27:43.250Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e1541fb1c881ac731cf6f93ebeb9d7a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.276Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-f4541085a4d7750691e1fac4: done: 0.436685 sec","job_status":"done","queue_duration_s":0.931588,"scheduling_latency_s":0.931637,"redis_calls":11,"redis_duration_s":0.004366,"redis_read_bytes":9,"redis_write_bytes":1292,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000729,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001922,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001715,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":130,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":130,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.059,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.352237,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00104956,"exclusive_lock_hold_duration_s":0.38915878,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.436685,"completed_at":"2026-08-15T13:27:44.644Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.093147,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.912092] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.647Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"db2a1b65b8c21d235185797a","created_at":"2026-08-15T13:27:43.290Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"058fd2a1204cda16190984dece135e6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.305Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-db2a1b65b8c21d235185797a: start","job_status":"start","queue_duration_s":1.341399,"scheduling_latency_s":1.341434} gitlab # [ 436.929480] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.671Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0} gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 436.934592] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"db2a1b65b8c21d235185797a","created_at":"2026-08-15T13:27:43.290Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"058fd2a1204cda16190984dece135e6f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.305Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-db2a1b65b8c21d235185797a: done: 0.036515 sec","job_status":"done","queue_duration_s":1.341399,"scheduling_latency_s":1.341434,"redis_calls":9,"redis_duration_s":0.002089,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000467,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00116,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000462,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019673,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036515,"completed_at":"2026-08-15T13:27:44.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004315,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 436.978599] gitaly[904]: time="2026-08-15T13:27:44.648Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:14.545" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:44.646" grpc.time_ms=1.187 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 436.991276] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:44.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"92422377c57b34f1a4afcddf","created_at":"2026-08-15T13:27:43.343Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f06f8f2cf3540ae8426051d3ce069368","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.362Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-92422377c57b34f1a4afcddf: start","job_status":"start","queue_duration_s":1.323003,"scheduling_latency_s":1.323031} gitlab # [ 437.005502] gitaly[904]: time="2026-08-15T13:27:44.752Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=15 command.spawn_token_fork_ms=2 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:54.203" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:44.734" grpc.time_ms=17.957 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.019311] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:44 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 201 670 "" "python-httpx/0.28.1" 308 gitlab # [ 437.022178] nixbot[3556]: {"ts": "2026-08-15T13:27:44.848Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e \"HTTP/1.1 201 Created\""} gitlab # [ 437.072709] gitaly[904]: time="2026-08-15T13:27:44.903Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRYQ87XV8HY1DDZN40GX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:14.802" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:44.902" grpc.time_ms=0.608 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.14 seconds) gitlab # [ 437.101403] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:44 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses HTTP/1.1" 200 672 "" "curl/8.21.0" 76 gitlab # [ 437.108517] gitaly[904]: time="2026-08-15T13:27:44.939Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRYK5MDCGZ0HJ1Y599H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:14.737" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:44.938" grpc.time_ms=0.554 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.171524] gitaly[904]: time="2026-08-15T13:27:45.002Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=575 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M02SPRYK5MDCGZ0HJ1Y599H0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:54.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:44.989" grpc.time_ms=12.604 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.248354] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 227 gitlab # [ 437.251092] nixbot[3556]: {"ts": "2026-08-15T13:27:45.080Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e \"HTTP/1.1 201 Created\""} gitlab # [ 437.289002] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.118Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"92422377c57b34f1a4afcddf","created_at":"2026-08-15T13:27:43.343Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f06f8f2cf3540ae8426051d3ce069368","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.362Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-92422377c57b34f1a4afcddf: done: 0.432662 sec","job_status":"done","queue_duration_s":1.323003,"scheduling_latency_s":1.323031,"redis_calls":564,"redis_duration_s":0.09074,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000687,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001368,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.088685,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.382124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.432662,"completed_at":"2026-08-15T13:27:45.118Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001043,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.362536] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.120Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"197545b838f635e0dffbd371","created_at":"2026-08-15T13:27:43.392Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6efded8b4217ab12b3eb2cd762bd246","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.423Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-197545b838f635e0dffbd371: start","job_status":"start","queue_duration_s":1.696636,"scheduling_latency_s":1.696666} gitlab # [ 437.381436] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"197545b838f635e0dffbd371","created_at":"2026-08-15T13:27:43.392Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b6efded8b4217ab12b3eb2cd762bd246","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.423Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-197545b838f635e0dffbd371: done: 0.025501 sec","job_status":"done","queue_duration_s":1.696636,"scheduling_latency_s":1.696666,"redis_calls":11,"redis_duration_s":0.0025610000000000003,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00048,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001087,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000994,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00054076,"exclusive_lock_hold_duration_s":0.00388546,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.025501,"completed_at":"2026-08-15T13:27:45.145Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002781,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.427781] gitaly[904]: time="2026-08-15T13:27:45.177Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPS63ZV78KGAETR554YXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-08-15T13:28:15.075" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-08-15T13:27:45.176" grpc.time_ms=0.992 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.440731] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.146Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8edc2456ced44bcf4e77ed92","created_at":"2026-08-15T13:27:43.443Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fe9466cd351074f7a0eb80039af889e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.460Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8edc2456ced44bcf4e77ed92: start","job_status":"start","queue_duration_s":1.686792,"scheduling_latency_s":1.686811} gitlab # [ 437.455036] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.168Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"8edc2456ced44bcf4e77ed92","created_at":"2026-08-15T13:27:43.443Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3fe9466cd351074f7a0eb80039af889e","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.460Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-8edc2456ced44bcf4e77ed92: done: 0.022015 sec","job_status":"done","queue_duration_s":1.686792,"scheduling_latency_s":1.686811,"redis_calls":11,"redis_duration_s":0.00176,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000553,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000342,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013956,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022015,"completed_at":"2026-08-15T13:27:45.168Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.001555,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.497762] gitaly[904]: time="2026-08-15T13:27:45.235Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=2 command.system_time_ms=3 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M02SPS63ZV78KGAETR554YXD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-08-15T13:27:55.175" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:45.226" grpc.time_ms=9.587 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 437.511304] nixbot[3556]: {"ts": "2026-08-15T13:27:45.320Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e \"HTTP/1.1 201 Created\""} gitlab # [ 437.514195] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:45 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/b0d1904a3636a352e89f8292bcc2601f40d8477e HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 227 gitlab # [ 437.516881] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.169Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"390886e0dac448a6a8ac18e0","created_at":"2026-08-15T13:27:43.473Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61aab23c7051d82eb29563456328bfaf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.496Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-390886e0dac448a6a8ac18e0: start","job_status":"start","queue_duration_s":1.673642,"scheduling_latency_s":1.673659} gitlab # [ 437.530292] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.289Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"390886e0dac448a6a8ac18e0","created_at":"2026-08-15T13:27:43.473Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"61aab23c7051d82eb29563456328bfaf","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.496Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-390886e0dac448a6a8ac18e0: done: 0.119456 sec","job_status":"done","queue_duration_s":1.673642,"scheduling_latency_s":1.673659,"redis_calls":70,"redis_duration_s":0.016063,"redis_read_bytes":118,"redis_write_bytes":42530,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001836,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000575,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37435,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.013264,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000388,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.098463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.119456,"completed_at":"2026-08-15T13:27:45.289Z","load_balancing_strategy":"primary","db_duration_s":0.00289,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.574380] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.290Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d90468d4345fe49105cf48cd","created_at":"2026-08-15T13:27:43.518Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1cbc2247d751ff43d09e8615ccc49b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:27:43.547Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d90468d4345fe49105cf48cd: start","job_status":"start","queue_duration_s":1.743331,"scheduling_latency_s":1.74335} gitlab # [ 437.597504] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.325Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"d90468d4345fe49105cf48cd","created_at":"2026-08-15T13:27:43.518Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b1cbc2247d751ff43d09e8615ccc49b4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-08-15T13:27:43.547Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-d90468d4345fe49105cf48cd: done: 0.034532 sec","job_status":"done","queue_duration_s":1.743331,"scheduling_latency_s":1.74335,"redis_calls":9,"redis_duration_s":0.00175,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00028,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000617,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020621,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034532,"completed_at":"2026-08-15T13:27:45.325Z","load_balancing_strategy":"primary","db_duration_s":0.00669,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.647937] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.326Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8d10acc9f0a345f3cb38c30d","created_at":"2026-08-15T13:27:43.575Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18b912bb6c7b50174e617e3f41247ebe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.593Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8d10acc9f0a345f3cb38c30d: start","job_status":"start","queue_duration_s":1.733223,"scheduling_latency_s":1.733243} gitlab # [ 437.664374] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.370Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"8d10acc9f0a345f3cb38c30d","created_at":"2026-08-15T13:27:43.575Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"18b912bb6c7b50174e617e3f41247ebe","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.593Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-8d10acc9f0a345f3cb38c30d: done: 0.043446 sec","job_status":"done","queue_duration_s":1.733223,"scheduling_latency_s":1.733243,"redis_calls":11,"redis_duration_s":0.002019,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000895,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000774,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021478,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00057922,"exclusive_lock_hold_duration_s":0.01339204,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.043446,"completed_at":"2026-08-15T13:27:45.370Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015513,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.710517] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.372Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ca5a0c2975437da0e01bb303","created_at":"2026-08-15T13:27:43.614Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e18eabcf63949224224b965d66bfd6fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.628Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ca5a0c2975437da0e01bb303: start","job_status":"start","queue_duration_s":1.743734,"scheduling_latency_s":1.743752} gitlab # [ 437.725183] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.382Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0} gitlab # [ 437.728698] nix-daemon[3765]: accepted connection from pid 4505, user nixbot gitlab # [ 437.729682] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ca5a0c2975437da0e01bb303","created_at":"2026-08-15T13:27:43.614Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e18eabcf63949224224b965d66bfd6fe","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.628Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ca5a0c2975437da0e01bb303: done: 0.018644 sec","job_status":"done","queue_duration_s":1.743734,"scheduling_latency_s":1.743752,"redis_calls":9,"redis_duration_s":0.0013809999999999998,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000311,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000296,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018644,"completed_at":"2026-08-15T13:27:45.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000426,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.773166] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.392Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f26e56cea755fd64b468076e","created_at":"2026-08-15T13:27:43.852Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f083cfe06cd358722e681572b0b579d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:27:43.861Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f26e56cea755fd64b468076e: start","job_status":"start","queue_duration_s":1.531136,"scheduling_latency_s":1.531154} gitlab # [ 437.786516] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.404Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0} gitlab # [ 437.789893] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.413Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"f26e56cea755fd64b468076e","created_at":"2026-08-15T13:27:43.852Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f083cfe06cd358722e681572b0b579d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-08-15T13:27:43.861Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-f26e56cea755fd64b468076e: done: 0.02144 sec","job_status":"done","queue_duration_s":1.531136,"scheduling_latency_s":1.531154,"redis_calls":9,"redis_duration_s":0.001836,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000409,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001035,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000392,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013161,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02144,"completed_at":"2026-08-15T13:27:45.413Z","load_balancing_strategy":"primary","db_duration_s":0.000447,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.831411] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.415Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b90111d35e43e2db981c3745","created_at":"2026-08-15T13:27:43.888Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a752980eae30d7a74094151692a29d11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.915Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b90111d35e43e2db981c3745: start","job_status":"start","queue_duration_s":1.5003,"scheduling_latency_s":1.500318} gitlab # [ 437.845974] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:45 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 88 gitlab # [ 437.848042] gitaly[904]: time="2026-08-15T13:27:45.620Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=3 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M02SPSKVZMJ7C3S4T3248YGP grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:45.605" grpc.time_ms=13.879 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 437.861213] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.434Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"b90111d35e43e2db981c3745","created_at":"2026-08-15T13:27:43.888Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a752980eae30d7a74094151692a29d11","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.915Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-b90111d35e43e2db981c3745: done: 0.019114 sec","job_status":"done","queue_duration_s":1.5003,"scheduling_latency_s":1.500318,"redis_calls":9,"redis_duration_s":0.001223,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000263,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000283,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011738,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019114,"completed_at":"2026-08-15T13:27:45.434Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000421,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.904244] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.435Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a195862c0a15c2b504321d70","created_at":"2026-08-15T13:27:43.935Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1599cdc7900d8364ae102313185242fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.949Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a195862c0a15c2b504321d70: start","job_status":"start","queue_duration_s":1.486108,"scheduling_latency_s":1.486126} gitlab # [ 437.918584] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.446Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 437.922251] gitaly[904]: time="2026-08-15T13:27:45.707Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPSPPA43SX6BTS8ANF2YV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:45.692" pid=904 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc gitlab # [ 437.932298] gitlab-workhorse[820]: time="2026-08-15T13:27:45Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=67 gitlab # [ 437.934142] gitlab-workhorse[820]: gitlab 2001:db8:1::1 - - [2026/08/15:13:27:45 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 85 gitlab # [ 437.936037] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a195862c0a15c2b504321d70","created_at":"2026-08-15T13:27:43.935Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1599cdc7900d8364ae102313185242fc","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:43.949Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a195862c0a15c2b504321d70: done: 0.017681 sec","job_status":"done","queue_duration_s":1.486108,"scheduling_latency_s":1.486126,"redis_calls":9,"redis_duration_s":0.001411,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00046,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00027,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010518,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.017681,"completed_at":"2026-08-15T13:27:45.453Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000339,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 437.979970] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b8367cbec9f74b770ad4c25e","created_at":"2026-08-15T13:27:44.000Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7453a7779a967b3975d0bd5f9d265638","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:27:44.024Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b8367cbec9f74b770ad4c25e: start","job_status":"start","queue_duration_s":1.42956,"scheduling_latency_s":1.429577} gitlab # [ 437.993781] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.468Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0} gitlab # [ 437.997881] gitaly[904]: time="2026-08-15T13:27:45.707Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPSPPA43SX6BTS8ANF2YV grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:45.692" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab # [ 438.011330] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.476Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"b8367cbec9f74b770ad4c25e","created_at":"2026-08-15T13:27:44.000Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7453a7779a967b3975d0bd5f9d265638","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-08-15T13:27:44.024Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-b8367cbec9f74b770ad4c25e: done: 0.02185 sec","job_status":"done","queue_duration_s":1.42956,"scheduling_latency_s":1.429577,"redis_calls":9,"redis_duration_s":0.0013189999999999999,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000299,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000739,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000281,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011764,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02185,"completed_at":"2026-08-15T13:27:45.476Z","load_balancing_strategy":"primary","db_duration_s":0.003424,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.052755] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.477Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"636e638d2b11b23b6fed0060","created_at":"2026-08-15T13:27:44.041Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f684c6198e94cbc262ec6867cfd0365","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.061Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-636e638d2b11b23b6fed0060: start","job_status":"start","queue_duration_s":1.416321,"scheduling_latency_s":1.416339} gitlab # [ 438.066925] nix-daemon[3765]: accepted connection from pid 4554, user nixbot gitlab # [ 438.067836] gitaly[904]: time="2026-08-15T13:27:45.708Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=10 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=11 command.spawn_token_fork_ms=1 command.system_time_ms=10 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M02SPSPPA43SX6BTS8ANF2YV grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-08-15T13:27:45.692" grpc.time_ms=14.678 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses gitlab # [ 438.082446] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.510Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"636e638d2b11b23b6fed0060","created_at":"2026-08-15T13:27:44.041Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f684c6198e94cbc262ec6867cfd0365","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.061Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-636e638d2b11b23b6fed0060: done: 0.033259 sec","job_status":"done","queue_duration_s":1.416321,"scheduling_latency_s":1.416339,"redis_calls":12,"redis_duration_s":0.002584,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000722,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001123,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000739,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02134,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00931478,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033259,"completed_at":"2026-08-15T13:27:45.510Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00497,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10} gitlab # [ 438.124576] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.511Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6327d320771164ce6642706c","created_at":"2026-08-15T13:27:44.092Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404be6ab4e550852a4ede897c7397b98","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.116Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6327d320771164ce6642706c: start","job_status":"start","queue_duration_s":1.395038,"scheduling_latency_s":1.395055} gitlab # [ 438.137270] gitaly[904]: time="2026-08-15T13:27:45.725Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:45.722" grpc.time_ms=2.187 pid=904 span.kind=server system=grpc gitlab # [ 438.146548] gitaly[904]: time="2026-08-15T13:27:45.753Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:45.752" grpc.time_ms=0.367 pid=904 span.kind=server system=grpc gitlab # [ 438.156302] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.537Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"6327d320771164ce6642706c","created_at":"2026-08-15T13:27:44.092Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"404be6ab4e550852a4ede897c7397b98","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.116Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-6327d320771164ce6642706c: done: 0.025849 sec","job_status":"done","queue_duration_s":1.395038,"scheduling_latency_s":1.395055,"redis_calls":9,"redis_duration_s":0.002,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000429,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001141,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00043,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025849,"completed_at":"2026-08-15T13:27:45.537Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004738,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.198963] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.538Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"661f99df036fe033d10f9c22","created_at":"2026-08-15T13:27:44.144Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41a2ac2d1051a30b7356b56fce45e450","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.170Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-661f99df036fe033d10f9c22: start","job_status":"start","queue_duration_s":1.368275,"scheduling_latency_s":1.368293} gitlab # [ 438.212997] gitaly[904]: time="2026-08-15T13:27:45.775Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-08-15T13:27:45.774" grpc.time_ms=0.497 pid=904 span.kind=server system=grpc gitlab # [ 438.222690] gitaly[904]: time="2026-08-15T13:27:45.781Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:15.690" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:45.691" operations="[repack_refs]" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab # [ 438.232590] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.573Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"661f99df036fe033d10f9c22","created_at":"2026-08-15T13:27:44.144Z","trace_propagation_headers":{"sentry-trace":"6f46e709a81548a5bb2f91ad8ee971c1-96baa07d067d45a8","baggage":"sentry-trace_id=6f46e709a81548a5bb2f91ad8ee971c1,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41a2ac2d1051a30b7356b56fce45e450","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.170Z","job_size_bytes":2,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-661f99df036fe033d10f9c22: done: 0.034897 sec","job_status":"done","queue_duration_s":1.368275,"scheduling_latency_s":1.368293,"redis_calls":18,"redis_duration_s":0.002483,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000619,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000186,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001244,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000434,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.034897,"completed_at":"2026-08-15T13:27:45.573Z","load_balancing_strategy":"primary","db_duration_s":0.001925,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60} gitlab # [ 438.274966] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.574Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e581e0ed77c3502c7976ce86","created_at":"2026-08-15T13:27:44.823Z","trace_propagation_headers":{"sentry-trace":"65f2a891b6314357994161da386725d6-cf806d39ec0a4287","baggage":"sentry-trace_id=65f2a891b6314357994161da386725d6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPRMSCFKE9SHNB53XG51K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.830Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e581e0ed77c3502c7976ce86: start","job_status":"start","queue_duration_s":0.744355,"scheduling_latency_s":0.744373} gitlab # [ 438.290805] gitaly[904]: time="2026-08-15T13:27:45.784Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:15.690" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:45.691" pid=904 remote_ip="2001:db8:1::1" repository_info="{false {6 703 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-08-15 13:26:00.27795458 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root gitlab # [ 438.302962] gitaly[904]: time="2026-08-15T13:27:45.787Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M02SPRMSCFKE9SHNB53XG51K diskcache=aa491491-c536-499b-ba83-9b92fe5de78a grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-08-15T13:28:15.690" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-08-15T13:27:45.691" pid=904 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/b0d1904a3636a352e89f8292bcc2601f40d8477e/statuses, in 0.22 seconds) (finished: subtest: eval and build post commit statuses, in 6.26 seconds) (finished: run the VM test script, in 511.30 seconds) gitlab # [ 438.313225] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.906Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e581e0ed77c3502c7976ce86","created_at":"2026-08-15T13:27:44.823Z","trace_propagation_headers":{"sentry-trace":"65f2a891b6314357994161da386725d6-cf806d39ec0a4287","baggage":"sentry-trace_id=65f2a891b6314357994161da386725d6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPRMSCFKE9SHNB53XG51K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.830Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3536,"job_status":"rescheduled","message":"PipelineProcessWorker JID-e581e0ed77c3502c7976ce86: rescheduled"} test script finished in 511.34s cleanup kill QemuMachine (pid 46) gitlab # [ 438.327621] sidekiq-cluster[3536]: {"severity":"INFO","time":"2026-08-15T13:27:45.914Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"e581e0ed77c3502c7976ce86","created_at":"2026-08-15T13:27:44.823Z","trace_propagation_headers":{"sentry-trace":"65f2a891b6314357994161da386725d6-cf806d39ec0a4287","baggage":"sentry-trace_id=65f2a891b6314357994161da386725d6,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M02SPRMSCFKE9SHNB53XG51K","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-08-15T13:27:44.830Z","job_size_bytes":3,"pid":3536,"sidekiq_tid":"m8qg","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-e581e0ed77c3502c7976ce86: done: 0.339315 sec","job_status":"done","queue_duration_s":0.744355,"scheduling_latency_s":0.744373,"gitaly_calls":3,"gitaly_duration_s":0.107084,"redis_calls":76,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.01296,"redis_read_bytes":168,"redis_write_bytes":21324,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001138,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001339,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.003069,"redis_queues_read_bytes":37,"redis_queues_write_bytes":14462,"redis_queues_metadata_calls":20,"redis_queues_metadata_duration_s":0.004204,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3301,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000284,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000448,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.00134,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.063,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_rqemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14) gitlab # e (finished: cleanup, in 1.39 seconds) additionally exposed symbols: gitlab, vlan1, start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh [] [{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-08-15T13:26:07.151Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': False, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [] [{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'evaluating', 'eval_warnings': None, 'error': None, 'created_at': '2026-08-15T13:27:39.133805Z', 'started_at': '2026-08-15T13:27:39.471363Z', 'finished_at': None}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluating flake', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (2 warnings)', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': '2026-08-15T13:27:44.802Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}] [{'id': 5, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded (2 warnings)', 'created_at': '2026-08-15T13:27:39.436Z', 'started_at': None, 'finished_at': '2026-08-15T13:27:44.802Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': 'b0d1904a3636a352e89f8292bcc2601f40d8477e', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-08-15T13:27:45.041Z', 'started_at': None, 'finished_at': '2026-08-15T13:27:45.288Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]