vm-test-run-nixbot-gitlab
checks.aarch64-linux.nixbot-gitlab
· build #403
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 3600 seconds6run the VM test script7start all VMs8gitlab: starting vm9gitlab # Disk image does not exist, creating the virtualisation disk image...10gitlab: QEMU running (pid 46)11gitlab # Formatting '/build/vm-state-gitlab/tmp.f72Jm1lTOw', fmt=raw size=107374182412gitlab # mke2fs 1.47.4 (6-Mar-2025)13gitlab # Discarding device blocks: 0/262144 done14gitlab # Creating filesystem with 262144 4k blocks and 65536 inodes15gitlab # Filesystem UUID: 981d9538-fe2e-4bd9-ba3f-58cd8945b95816gitlab # Superblock backups stored on blocks:17(finished: start all VMs, in 67.42 seconds)18gitlab # 32768, 98304, 163840, 22937619gitlab: waiting for unit setup-gitlab.service20gitlab # 21gitlab: waiting for the VM to finish booting22gitlab # Allocating group tables: 0/8 done23gitlab # Writing inode tables: 0/8 done24gitlab # Creating journal (8192 blocks): done25gitlab # Writing superblocks and filesystem accounting information: 0/8 done26gitlab # 27gitlab # Virtualisation disk image created.28gitlab # Creating Nix store image...29gitlab # Created Nix store image.30gitlab # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]31gitlab # [ 0.000000] Linux version 6.18.43 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Thu Aug 6 17:27:09 UTC 202632gitlab # [ 0.000000] KASLR enabled33gitlab # [ 0.000000] random: crng init done34gitlab # [ 0.000000] Machine model: linux,dummy-virt35gitlab # [ 0.000000] efi: UEFI not found.36gitlab # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT37gitlab # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x00000001bfffffff]38gitlab # [ 0.000000] NODE_DATA(0) allocated [mem 0x1bfdeb740-0x1bfdeeebf]39gitlab # [ 0.000000] Zone ranges:40gitlab # [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]41gitlab # [ 0.000000] DMA32 empty42gitlab # [ 0.000000] Normal [mem 0x0000000100000000-0x00000001bfffffff]43gitlab # [ 0.000000] Device empty44gitlab # [ 0.000000] Movable zone start for each node45gitlab # [ 0.000000] Early memory node ranges46gitlab # [ 0.000000] node 0: [mem 0x0000000040000000-0x00000001bfffffff]47gitlab # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000001bfffffff]48gitlab # [ 0.000000] cma: Reserved 32 MiB at 0x00000000fe00000049gitlab # [ 0.000000] psci: probing for conduit method from DT.50gitlab # [ 0.000000] psci: PSCIv1.3 detected in firmware.51gitlab # [ 0.000000] psci: Using standard PSCI v0.2 function IDs52gitlab # [ 0.000000] psci: Trusted OS migration not required53gitlab # [ 0.000000] psci: SMC Calling Convention v1.154gitlab # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)55gitlab # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u31129656gitlab # [ 0.000000] Detected PIPT I-cache on CPU057gitlab # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)58gitlab # [ 0.000000] CPU features: detected: GICv3 CPU interface59gitlab # [ 0.000000] CPU features: detected: Spectre-v460gitlab # [ 0.000000] CPU features: detected: Spectre-BHB61gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_3862gitlab # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_2363gitlab # [ 0.000000] alternatives: applying boot alternatives64gitlab # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2hpmbb80a6jj3dmanfcy6yy2az91lf6i-nixos-system-gitlab-test/init regInfo=/nix/store/q9zb6cdmv2q8ihlr8cga7p4qy6gjvcyl-closure-info/registration console=ttyAMA0,115200n8 console=tty065gitlab # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/q9zb6cdmv2q8ihlr8cga7p4qy6gjvcyl-closure-info/registration", will be passed to user space.66gitlab # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes67gitlab # [ 0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes, linear)68gitlab # [ 0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)69gitlab # [ 0.000000] software IO TLB: area num 4.70gitlab # [ 0.000000] software IO TLB: mapped [mem 0x00000000fa000000-0x00000000fe000000] (64MB)71gitlab # [ 0.000000] Fallback order for Node 0: 072gitlab # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 157286473gitlab # [ 0.000000] Policy zone: Normal74gitlab # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off75gitlab # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=176gitlab # [ 0.000000] allocated 12582912 bytes of page_ext77gitlab # [ 0.000000] ftrace: allocating 74867 entries in 294 pages78gitlab # [ 0.000000] ftrace: allocated 294 pages with 4 groups79gitlab # [ 0.000000] rcu: Hierarchical RCU implementation.80gitlab # [ 0.000000] rcu: RCU event tracing is enabled.81gitlab # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=4.82gitlab # [ 0.000000] Trampoline variant of Tasks RCU enabled.83gitlab # [ 0.000000] Rude variant of Tasks RCU enabled.84gitlab # [ 0.000000] Tracing variant of Tasks RCU enabled.85gitlab # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.86gitlab # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=487gitlab # [ 0.000000] RCU Tasks: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.88gitlab # [ 0.000000] RCU Tasks Rude: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.89gitlab # [ 0.000000] RCU Tasks Trace: Setting shift to 2 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=4.90gitlab # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 091gitlab # [ 0.000000] GICv3: 256 SPIs implemented92gitlab # [ 0.000000] GICv3: 0 Extended SPIs implemented93gitlab # [ 0.000000] Root IRQ handler: gic_handle_irq94gitlab # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI95gitlab # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=096gitlab # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a000097gitlab # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]98gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @100f10000 (indirect, esz 8, psz 64K, shr 1)99gitlab # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @100f20000 (flat, esz 8, psz 64K, shr 1)100gitlab # [ 0.000000] GICv3: using LPI property table @0x0000000100f40000101gitlab # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000100f50000102gitlab # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.103gitlab # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns104gitlab # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).105gitlab # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns106gitlab # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns107gitlab # [ 0.000032] arm-pv: using stolen time PV108gitlab # [ 0.000388] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)109gitlab # [ 0.000618] Console: colour dummy device 80x25110gitlab # [ 0.000626] printk: legacy console [tty0] enabled111gitlab # [ 0.000822] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)112gitlab # [ 0.000828] pid_max: default: 32768 minimum: 301113gitlab # [ 0.000941] LSM: initializing lsm=capability,landlock,yama,bpf,ima114gitlab # [ 0.001076] landlock: Up and running.115gitlab # [ 0.001079] Yama: becoming mindful.116gitlab # [ 0.001510] LSM support for eBPF active117gitlab # [ 0.001716] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)118gitlab # [ 0.001846] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes, linear)119gitlab # [ 0.003115] cacheinfo: Unable to detect cache hierarchy for CPU 0120gitlab # [ 0.003854] rcu: Hierarchical SRCU implementation.121gitlab # [ 0.003858] rcu: Max phase no-delay instances is 1000.122gitlab # [ 0.004049] Timer migration: 1 hierarchy levels; 8 children per group; 1 crossnode level123gitlab # [ 0.005195] fsl-mc MSI: its@8080000 domain created124gitlab # [ 0.005290] EFI services will not be available.125gitlab # [ 0.005506] smp: Bringing up secondary CPUs ...126gitlab # [ 0.006034] Detected PIPT I-cache on CPU1127gitlab # [ 0.006096] GICv3: CPU1: found redistributor 1 region 0:0x00000000080c0000128gitlab # [ 0.006169] GICv3: CPU1: using allocated LPI pending table @0x0000000100f60000129gitlab # [ 0.006240] CPU1: Booted secondary processor 0x0000000001 [0xc00fac40]130gitlab # [ 0.007095] Detected PIPT I-cache on CPU2131gitlab # [ 0.007157] GICv3: CPU2: found redistributor 2 region 0:0x00000000080e0000132gitlab # [ 0.007230] GICv3: CPU2: using allocated LPI pending table @0x0000000100f70000133gitlab # [ 0.007301] CPU2: Booted secondary processor 0x0000000002 [0xc00fac40]134gitlab # [ 0.008096] Detected PIPT I-cache on CPU3135gitlab # [ 0.008163] GICv3: CPU3: found redistributor 3 region 0:0x0000000008100000136gitlab # [ 0.008234] GICv3: CPU3: using allocated LPI pending table @0x0000000100f80000137gitlab # [ 0.008304] CPU3: Booted secondary processor 0x0000000003 [0xc00fac40]138gitlab # [ 0.008601] smp: Brought up 1 node, 4 CPUs139gitlab # [ 0.008627] SMP: Total of 4 processors activated.140gitlab # [ 0.008630] CPU: All CPU(s) started at EL1141gitlab # [ 0.008638] CPU features: detected: Branch Target Identification142gitlab # [ 0.008642] CPU features: detected: ARMv8.4 Translation Table Level143gitlab # [ 0.008645] CPU features: detected: Instruction cache invalidation not required for I/D coherence144gitlab # [ 0.008648] CPU features: detected: Data cache clean to the PoU not required for I/D coherence145gitlab # [ 0.008651] CPU features: detected: Common not Private translations146gitlab # [ 0.008654] CPU features: detected: CRC32 instructions147gitlab # [ 0.008657] CPU features: detected: Data cache clean to Point of Deep Persistence148gitlab # [ 0.008660] CPU features: detected: Data cache clean to Point of Persistence149gitlab # [ 0.008663] CPU features: detected: Data independent timing control (DIT)150gitlab # [ 0.008665] CPU features: detected: E0PD151gitlab # [ 0.008667] CPU features: detected: Enhanced Counter Virtualization152gitlab # [ 0.008670] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)153gitlab # [ 0.008673] CPU features: detected: Enhanced Virtualization Traps154gitlab # [ 0.008676] CPU features: detected: Fine Grained Traps155gitlab # [ 0.008679] CPU features: detected: Generic authentication (architected QARMA5 algorithm)156gitlab # [ 0.008682] CPU features: detected: RCpc load-acquire (LDAPR)157gitlab # [ 0.008685] CPU features: detected: LSE atomic instructions158gitlab # [ 0.008687] CPU features: detected: Privileged Access Never159gitlab # [ 0.008690] CPU features: detected: PMUv3160gitlab # [ 0.008692] CPU features: detected: RAS Extension Support161gitlab # [ 0.008694] CPU features: detected: RASv1p1 Extension Support162gitlab # [ 0.008697] CPU features: detected: Random Number Generator163gitlab # [ 0.008699] CPU features: detected: Speculation barrier (SB)164gitlab # [ 0.008702] CPU features: detected: Stage-2 Force Write-Back165gitlab # [ 0.008704] CPU features: detected: TLB range maintenance instructions166gitlab # [ 0.008708] CPU features: detected: Speculative Store Bypassing Safe (SSBS)167gitlab # [ 0.008800] alternatives: applying system-wide alternatives168gitlab # [ 0.011781] CPU features: detected: BBM Level 2 without TLB conflict abort169gitlab # [ 0.012021] Memory: 5963744K/6291456K available (24448K kernel code, 7090K rwdata, 26340K rodata, 4736K init, 1106K bss, 272908K reserved, 32768K cma-reserved)170gitlab # [ 0.013640] devtmpfs: initialized171gitlab # [ 0.017126] posixtimers hash table entries: 2048 (order: 3, 32768 bytes, linear)172gitlab # [ 0.017206] futex hash table entries: 1024 (65536 bytes on 1 NUMA nodes, total 64 KiB, linear).173gitlab # [ 0.017392] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL174gitlab # [ 0.017396] 0 pages in range for non-PLT usage175gitlab # [ 0.017398] 508288 pages in range for PLT usage176gitlab # [ 0.017519] pinctrl core: initialized pinctrl subsystem177gitlab # [ 0.018324] DMI not present or invalid.178gitlab # [ 0.021975] NET: Registered PF_NETLINK/PF_ROUTE protocol family179gitlab # [ 0.024510] DMA: preallocated 1024 KiB GFP_KERNEL pool for atomic allocations180gitlab # [ 0.024791] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations181gitlab # [ 0.026713] DMA: preallocated 1024 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations182gitlab # [ 0.026735] audit: initializing netlink subsys (disabled)183gitlab # [ 0.026924] audit: type=2000 audit(0.024:1): state=initialized audit_enabled=0 res=1184gitlab # [ 0.027425] thermal_sys: Registered thermal governor 'fair_share'185gitlab # [ 0.027428] thermal_sys: Registered thermal governor 'bang_bang'186gitlab # [ 0.027432] thermal_sys: Registered thermal governor 'step_wise'187gitlab # [ 0.027434] thermal_sys: Registered thermal governor 'user_space'188gitlab # [ 0.027437] thermal_sys: Registered thermal governor 'power_allocator'189gitlab # [ 0.027492] cpuidle: using governor ladder190gitlab # [ 0.027511] cpuidle: using governor menu191gitlab # [ 0.027712] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.192gitlab # [ 0.027846] ASID allocator initialised with 65536 entries193gitlab # [ 0.029091] Serial: AMBA PL011 UART driver194gitlab # [ 0.034174] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1195gitlab # [ 0.034366] printk: console [ttyAMA0] enabled196gitlab # [ 0.040095] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages197gitlab # [ 0.040108] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page198gitlab # [ 0.040113] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages199gitlab # [ 0.040117] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page200gitlab # [ 0.040122] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages201gitlab # [ 0.040126] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page202gitlab # [ 0.040130] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages203gitlab # [ 0.040134] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page204gitlab # [ 0.047368] fbcon: Taking over console205gitlab # [ 0.047380] ACPI: Interpreter disabled.206gitlab # [ 0.048305] iommu: Default domain type: Translated207gitlab # [ 0.048313] iommu: DMA domain TLB invalidation policy: strict mode208gitlab # [ 0.062474] SCSI subsystem initialized209gitlab # [ 0.063090] usbcore: registered new interface driver usbfs210gitlab # [ 0.063152] usbcore: registered new interface driver hub211gitlab # [ 0.063178] usbcore: registered new device driver usb212gitlab # [ 0.063677] pps_core: LinuxPPS API ver. 1 registered213gitlab # [ 0.063682] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>214gitlab # [ 0.063694] PTP clock support registered215gitlab # [ 0.063768] EDAC MC: Ver: 3.0.0216gitlab # [ 0.064160] scmi_core: SCMI protocol bus registered217gitlab # [ 0.064768] FPGA manager framework218gitlab # [ 0.065938] vgaarb: loaded219gitlab # [ 0.066899] clocksource: Switched to clocksource arch_sys_counter220gitlab # [ 0.070261] VFS: Disk quotas dquot_6.6.0221gitlab # [ 0.070297] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)222gitlab # [ 0.070635] netfs: FS-Cache loaded223gitlab # [ 0.070748] pnp: PnP ACPI: disabled224gitlab # [ 0.076014] NET: Registered PF_INET protocol family225gitlab # [ 0.077550] IP idents hash table entries: 131072 (order: 8, 1048576 bytes, linear)226gitlab # [ 0.119521] tcp_listen_portaddr_hash hash table entries: 4096 (order: 4, 65536 bytes, linear)227gitlab # [ 0.119592] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)228gitlab # [ 0.119653] TCP established hash table entries: 65536 (order: 7, 524288 bytes, linear)229gitlab # [ 0.120053] TCP bind hash table entries: 65536 (order: 9, 2097152 bytes, linear)230gitlab # [ 0.120899] TCP: Hash tables configured (established 65536 bind 65536)231gitlab # [ 0.121017] MPTCP token hash table entries: 8192 (order: 6, 196608 bytes, linear)232gitlab # [ 0.121082] UDP hash table entries: 4096 (order: 6, 262144 bytes, linear)233gitlab # [ 0.121252] UDP-Lite hash table entries: 4096 (order: 6, 262144 bytes, linear)234gitlab # [ 0.121482] NET: Registered PF_UNIX/PF_LOCAL protocol family235gitlab # [ 0.121547] NET: Registered PF_XDP protocol family236gitlab # [ 0.121566] PCI: CLS 0 bytes, default 64237gitlab # [ 0.121782] Trying to unpack rootfs image as initramfs...238gitlab # [ 0.127241] kvm [1]: HYP mode not available239gitlab # [ 0.181768] Initialise system trusted keyrings240gitlab # [ 0.181975] workingset: timestamp_bits=42 max_order=21 bucket_order=0241gitlab # [ 0.182597] squashfs: version 4.0 (2009/01/31) Phillip Lougher242gitlab # [ 0.182674] 9p: Installing v9fs 9p2000 file system support243gitlab # [ 0.201764] Key type asymmetric registered244gitlab # [ 0.201770] Asymmetric key parser 'x509' registered245gitlab # [ 0.201839] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)246gitlab # [ 0.202011] io scheduler mq-deadline registered247gitlab # [ 0.202019] io scheduler kyber registered248gitlab # [ 0.212781] pl061_gpio 9030000.pl061: PL061 GPIO chip registered249gitlab # [ 0.219002] ledtrig-cpu: registered to indicate activity on CPUs250gitlab # [ 0.219554] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:251gitlab # [ 0.219572] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000252gitlab # [ 0.219599] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000253gitlab # [ 0.219609] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000254gitlab # [ 0.219635] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits255gitlab # [ 0.219662] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]256gitlab # [ 0.219764] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00257gitlab # [ 0.219774] pci_bus 0000:00: root bus resource [bus 00-ff]258gitlab # [ 0.219779] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]259gitlab # [ 0.219784] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]260gitlab # [ 0.219789] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]261gitlab # [ 0.219889] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint262gitlab # [ 0.220662] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint263gitlab # [ 0.221047] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]264gitlab # [ 0.221061] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]265gitlab # [ 0.221088] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]266gitlab # [ 0.221100] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]267gitlab # [ 0.221637] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint268gitlab # [ 0.221864] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]269gitlab # [ 0.221876] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]270gitlab # [ 0.221903] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]271gitlab # [ 0.222470] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint272gitlab # [ 0.222698] pci 0000:00:03.0: BAR 0 [io 0x0000-0x001f]273gitlab # [ 0.222709] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]274gitlab # [ 0.222736] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]275gitlab # [ 0.223331] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint276gitlab # [ 0.223542] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]277gitlab # [ 0.223554] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]278gitlab # [ 0.223581] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]279gitlab # [ 0.224127] pci 0000:00:05.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint280gitlab # [ 0.224350] pci 0000:00:05.0: BAR 0 [io 0x0000-0x007f]281gitlab # [ 0.224362] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]282gitlab # [ 0.224388] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]283gitlab # [ 0.224958] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint284gitlab # [ 0.225178] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]285gitlab # [ 0.225190] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]286gitlab # [ 0.225220] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]287gitlab # [ 0.225791] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint288gitlab # [ 0.226007] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]289gitlab # [ 0.226018] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]290gitlab # [ 0.226044] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]291gitlab # [ 0.226057] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]292gitlab # [ 0.226617] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint293gitlab # [ 0.226840] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]294gitlab # [ 0.226867] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]295gitlab # [ 0.227428] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint296gitlab # [ 0.227663] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]297gitlab # [ 0.227690] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]298gitlab # [ 0.228158] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint299gitlab # [ 0.228378] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]300gitlab # [ 0.228664] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint301gitlab # [ 0.229036] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]302gitlab # [ 0.229049] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]303gitlab # [ 0.229075] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]304gitlab # [ 0.229646] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint305gitlab # [ 0.229869] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]306gitlab # [ 0.229881] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]307gitlab # [ 0.229907] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]308gitlab # [ 0.230599] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned309gitlab # [ 0.230605] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned310gitlab # [ 0.230609] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned311gitlab # [ 0.230657] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned312gitlab # [ 0.230701] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned313gitlab # [ 0.230754] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned314gitlab # [ 0.230807] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned315gitlab # [ 0.230857] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned316gitlab # [ 0.230927] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned317gitlab # [ 0.230980] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned318gitlab # [ 0.231029] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned319gitlab # [ 0.231073] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned320gitlab # [ 0.231145] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned321gitlab # [ 0.231189] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned322gitlab # [ 0.231206] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned323gitlab # [ 0.231223] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned324gitlab # [ 0.231240] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned325gitlab # [ 0.231258] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned326gitlab # [ 0.231276] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned327gitlab # [ 0.231298] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned328gitlab # [ 0.231316] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned329gitlab # [ 0.231334] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned330gitlab # [ 0.231355] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned331gitlab # [ 0.231374] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned332gitlab # [ 0.231397] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned333gitlab # [ 0.231420] pci 0000:00:05.0: BAR 0 [io 0x1000-0x107f]: assigned334gitlab # [ 0.231437] pci 0000:00:06.0: BAR 0 [io 0x1080-0x10ff]: assigned335gitlab # [ 0.231459] pci 0000:00:0b.0: BAR 0 [io 0x1100-0x113f]: assigned336gitlab # [ 0.231476] pci 0000:00:01.0: BAR 0 [io 0x1140-0x115f]: assigned337gitlab # [ 0.231493] pci 0000:00:02.0: BAR 0 [io 0x1160-0x117f]: assigned338gitlab # [ 0.231510] pci 0000:00:03.0: BAR 0 [io 0x1180-0x119f]: assigned339gitlab # [ 0.231528] pci 0000:00:04.0: BAR 0 [io 0x11a0-0x11bf]: assigned340gitlab # [ 0.231550] pci 0000:00:07.0: BAR 0 [io 0x11c0-0x11df]: assigned341gitlab # [ 0.231567] pci 0000:00:0c.0: BAR 0 [io 0x11e0-0x11ff]: assigned342gitlab # [ 0.231589] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]343gitlab # [ 0.231594] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]344gitlab # [ 0.231597] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]345gitlab # [ 0.233056] pci 0000:00:0a.0: enabling device (0000 -> 0002)346gitlab # [ 0.245062] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)347gitlab # [ 0.247596] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)348gitlab # [ 0.249718] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)349gitlab # [ 0.251949] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)350gitlab # [ 0.254206] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)351gitlab # [ 0.258935] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)352gitlab # [ 0.261548] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)353gitlab # [ 0.264358] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)354gitlab # [ 0.267001] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)355gitlab # [ 0.270835] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)356gitlab # [ 0.273928] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)357gitlab # [ 0.280113] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled358gitlab # [ 0.282026] msm_serial: driver initialized359gitlab # [ 0.282201] SuperH (H)SCI(F) driver initialized360gitlab # [ 0.282261] STM32 USART driver initialized361gitlab # [ 0.306885] loop: module loaded362gitlab # [ 0.307129] virtio_blk virtio4: 4/0/0 default/read/poll queues363gitlab # [ 0.309337] virtio_blk virtio4: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)364gitlab # [ 0.311804] virtio_blk virtio5: 4/0/0 default/read/poll queues365gitlab # [ 0.313836] virtio_blk virtio5: [vdb] 12285672 512-byte logical blocks (6.29 GB/5.86 GiB)366gitlab # [ 0.316161] megasas: 07.734.00.00-rc1367gitlab # [ 0.316965] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]368gitlab # [ 0.319032] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000369gitlab # [ 0.319050] Intel/Sharp Extended Query Table at 0x0031370gitlab # [ 0.320886] Using buffer write method371gitlab # [ 0.320919] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]372gitlab # [ 0.322836] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x000000373gitlab # [ 0.322902] Intel/Sharp Extended Query Table at 0x0031374gitlab # [ 0.324886] Using buffer write method375gitlab # [ 0.324902] Concatenating MTD devices:376gitlab # [ 0.324904] (0): "0.flash"377gitlab # [ 0.324907] (1): "0.flash"378gitlab # [ 0.324909] into device "0.flash"379gitlab # [ 0.543717] Freeing initrd memory: 27184K380gitlab # [ 0.554590] tun: Universal TUN/TAP device driver, 1.6381gitlab # [ 0.559490] thunder_xcv, ver 1.0382gitlab # [ 0.559563] thunder_bgx, ver 1.0383gitlab # [ 0.559594] nicpf, ver 1.0384gitlab # [ 0.560485] e1000: Intel(R) PRO/1000 Network Driver385gitlab # [ 0.560491] e1000: Copyright (c) 1999-2006 Intel Corporation.386gitlab # [ 0.560525] e1000e: Intel(R) PRO/1000 Network Driver387gitlab # [ 0.560533] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.388gitlab # [ 0.560585] igb: Intel(R) Gigabit Ethernet Network Driver389gitlab # [ 0.560590] igb: Copyright (c) 2007-2014 Intel Corporation.390gitlab # [ 0.560623] igbvf: Intel(R) Gigabit Virtual Function Network Driver391gitlab # [ 0.560629] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.392gitlab # [ 0.560899] sky2: driver version 1.30393gitlab # [ 0.563451] usbcore: registered new interface driver usb-storage394gitlab # [ 0.563572] usbcore: registered new interface driver usbserial_generic395gitlab # [ 0.563589] usbserial: USB Serial support registered for generic396gitlab # [ 0.563628] ehci-pci 0000:00:0a.0: EHCI Host Controller397gitlab # [ 0.563659] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 1398gitlab # [ 0.563945] ehci-pci 0000:00:0a.0: irq 15, io mem 0x10089000399gitlab # [ 0.564503] hv_vmbus: registering driver hyperv_keyboard400gitlab # [ 0.566088] rtc-pl031 9010000.pl031: registered as rtc0401gitlab # [ 0.566124] rtc-pl031 9010000.pl031: setting system clock to 2026-10-02T16:52:35 UTC (1790959955)402gitlab # [ 0.566611] i2c_dev: i2c /dev entries driver403gitlab # [ 0.571004] sdhci: Secure Digital Host Controller Interface driver404gitlab # [ 0.571011] sdhci: Copyright(c) Pierre Ossman405gitlab # [ 0.571440] Synopsys Designware Multimedia Card Interface Driver406gitlab # [ 0.572018] sdhci-pltfm: SDHCI platform and OF driver helper407gitlab # [ 0.574621] hid: raw HID events driver (C) Jiri Kosina408gitlab # [ 0.574992] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.00409gitlab # [ 0.575088] usbcore: registered new interface driver usbhid410gitlab # [ 0.575095] usbhid: USB HID core driver411gitlab # [ 0.575750] hub 1-0:1.0: USB hub found412gitlab # [ 0.575872] hub 1-0:1.0: 6 ports detected413gitlab # [ 0.597278] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available414gitlab # [ 0.599734] drop_monitor: Initializing network drop monitor service415gitlab # [ 0.599957] NET: Registered PF_INET6 protocol family416gitlab # [ 0.601398] Segment Routing with IPv6417gitlab # [ 0.601433] In-situ OAM (IOAM) with IPv6418gitlab # [ 0.601503] NET: Registered PF_PACKET protocol family419gitlab # [ 0.601691] 9pnet: Installing 9P2000 support420gitlab # [ 0.604164] Key type dns_resolver registered421gitlab # [ 0.614407] registered taskstats version 1422gitlab # [ 0.614744] Loading compiled-in X.509 certificates423gitlab # [ 0.629554] Demotion targets for Node 0: null424gitlab # [ 0.629744] Key type .fscrypt registered425gitlab # [ 0.629749] Key type fscrypt-provisioning registered426gitlab # [ 0.629884] ima: No TPM chip found, activating TPM-bypass!427gitlab # [ 0.629904] ima: Allocated hash algorithm: sha1428gitlab # [ 0.629942] ima: No architecture policies found429gitlab # [ 0.630958] input: gpio-keys as /devices/platform/gpio-keys/input/input0430gitlab # [ 0.655725] clk: Disabling unused clocks431gitlab # [ 0.655763] PM: genpd: Disabling unused power domains432gitlab # [ 0.660335] Freeing unused kernel memory: 4736K433gitlab # [ 0.660601] Run /init as init process434gitlab # [ 0.687257] systemd[1]: Successfully made /usr/ read-only.435gitlab # [ 0.822945] usb 1-1: new high-speed USB device number 2 using ehci-pci436gitlab # [ 0.972741] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input1437gitlab # [ 1.021629] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)438gitlab # [ 1.021671] systemd[1]: Detected virtualization qemu.439gitlab # [ 1.021716] systemd[1]: Detected architecture arm64.440gitlab # [ 1.021725] systemd[1]: Running in initrd.441gitlab # [ 1.022220] systemd[1]: Initializing machine ID from random generator.442gitlab # [ 1.022439] systemd[1]: Hostname set to <gitlab>.443gitlab # [ 1.087307] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input0444gitlab # [ 1.207005] usb 1-2: new high-speed USB device number 3 using ehci-pci445gitlab # [ 1.223730] systemd[1]: bpf-restrict-fs: LSM BPF program attached446gitlab # [ 1.306732] systemd[1]: Queued start job for default target Initrd Default Target.447gitlab # [ 1.337000] systemd[1]: Created slice Slice /system/modprobe.448gitlab # [ 1.337573] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.449gitlab # [ 1.337671] systemd[1]: Expecting device /dev/disk/by-label/nix-store...450gitlab # [ 1.337689] systemd[1]: Expecting device /dev/disk/by-label/nixos...451gitlab # [ 1.337798] systemd[1]: Reached target Path Units.452gitlab # [ 1.337854] systemd[1]: Reached target Slice Units.453gitlab # [ 1.337958] systemd[1]: Reached target Swaps.454gitlab # [ 1.337982] systemd[1]: Reached target Timer Units.455gitlab # [ 1.338338] systemd[1]: Listening on D-Bus System Message Bus Socket.456gitlab # [ 1.338749] systemd[1]: Listening on Journal Socket (/dev/log).457gitlab # [ 1.339169] systemd[1]: Listening on Journal Sockets.458gitlab # [ 1.339579] systemd[1]: Listening on udev Control Socket.459gitlab # [ 1.339802] systemd[1]: Listening on udev Kernel Socket.460gitlab # [ 1.339869] systemd[1]: Reached target Socket Units.461gitlab # [ 1.343180] systemd[1]: Starting Create List of Static Device Nodes...462gitlab # [ 1.346381] systemd[1]: Starting Load Kernel Module 9pnet_virtio...463gitlab # [ 1.346596] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs464gitlab # [ 1.351173] systemd[1]: Mounting Kernel Configuration File System...465gitlab # [ 1.361032] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input2466gitlab # [ 1.361457] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input0467gitlab # [ 1.371193] systemd[1]: Starting Journal Service...468gitlab # [ 1.379187] systemd[1]: Starting Load Kernel Modules...469gitlab # [ 1.379422] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os470gitlab # [ 1.383015] systemd[1]: Starting Coldplug All udev Devices...471gitlab # [ 1.386036] systemd[1]: Finished Create List of Static Device Nodes.472gitlab # [ 1.393325] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.473gitlab # [ 1.394268] systemd[1]: Finished Load Kernel Module 9pnet_virtio.474gitlab # [ 1.395301] systemd[1]: Mounted Kernel Configuration File System.475gitlab # [ 1.400256] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...476gitlab # [ 1.420985] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.477gitlab # [ 1.423025] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev478gitlab # [ 1.429574] [drm] pci: virtio-gpu-pci detected at 0000:00:08.0479gitlab # [ 1.429802] [drm] features: -virgl +edid -resource_blob -host_visible480gitlab # [ 1.429824] [drm] features: -context_init481gitlab # [ 1.434313] [drm] number of scanouts: 1482gitlab # [ 1.434360] [drm] number of cap sets: 0483gitlab # [ 1.435341] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic484gitlab # [ 1.435357] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 0485gitlab # [ 1.440530] systemd-journald[94]: Collecting audit messages is disabled.486gitlab # [ 1.443408] Console: switching to colour frame buffer device 160x50487gitlab # [ 1.447450] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.488gitlab # [ 1.448327] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device489gitlab # [ 1.450542] systemd[1]: Starting Create Static Device Nodes in /dev...490gitlab # [ 1.461822] systemd[1]: Finished Load Kernel Modules.491gitlab # [ 1.471175] systemd[1]: Starting Apply Kernel Variables...492gitlab # [ 1.478202] systemd[1]: Finished Create Static Device Nodes in /dev.493gitlab # [ 1.478731] systemd[1]: Reached target Preparation for Local File Systems.494gitlab # [ 1.479495] systemd[1]: Reached target Local File Systems.495gitlab # [ 1.491142] systemd[1]: Starting Rule-based Manager for Device Events and Files...496gitlab # [ 1.500046] systemd[1]: Finished Apply Kernel Variables.497gitlab # [ 1.547519] systemd[1]: Started Journal Service.498gitlab # [ 1.544675] systemd-modules-load[96]: Using 4 probe threads499gitlab # [ 1.546817] systemd-modules-load[96]: Module 'virtio_balloon' is built in500gitlab # [ 1.548860] systemd-modules-load[96]: Module 'virtio_console' is built in501gitlab # [ 1.550020] systemd-modules-load[96]: Module 'virtio_rng' is built in502gitlab # [ 1.551519] systemd-modules-load[96]: Inserted module 'dm_mod'503gitlab # [ 1.552581] systemd-modules-load[96]: Inserted module 'virtio_gpu'504gitlab # [ 1.553588] systemd[1]: Starting Create System Files and Directories...505gitlab # [ 1.555377] systemd-udevd[106]: Using default interface naming scheme 'v261'.506gitlab # [ 1.590321] systemd[1]: Started Rule-based Manager for Device Events and Files.507gitlab # [ 1.607191] systemd[1]: Finished Create System Files and Directories.508gitlab # [ 1.620491] systemd[1]: Starting Virtual Console Setup...509gitlab # [ 1.687689] systemd-vconsole-setup[129]: Configuration of first virtual console was skipped, ignoring remaining ones.510gitlab # [ 1.690940] systemd[1]: Finished Virtual Console Setup.511gitlab # [ 2.341573] systemd[1]: Finished Coldplug All udev Devices.512gitlab # [ 2.343199] systemd[1]: Reached target System Initialization.513gitlab # [ 2.344655] systemd[1]: Reached target Basic System.514gitlab # [ 2.466687] (udev-worker)[123]: Network interface NamePolicy= disabled on kernel command line.515gitlab # [ 2.496669] (udev-worker)[116]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.516gitlab # [ 2.501246] (udev-worker)[116]: Network interface NamePolicy= disabled on kernel command line.517gitlab # [ 2.529959] systemd[1]: Found device /dev/disk/by-label/nixos.518gitlab # [ 2.531340] systemd[1]: Reached target Initrd Root Device.519gitlab # [ 2.533433] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...520gitlab # [ 2.540677] systemd[1]: Found device /dev/disk/by-label/nix-store.521gitlab # [ 2.573838] systemd-fsck[151]: nixos: clean, 12/65536 files, 13019/262144 blocks522gitlab # [ 2.579451] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.523gitlab # [ 2.587940] systemd[1]: Mounting /sysroot...524gitlab # [ 2.624012] EXT4-fs (vda): mounted filesystem 981d9538-fe2e-4bd9-ba3f-58cd8945b958 r/w with ordered data mode. Quota mode: none.525gitlab # [ 2.623450] systemd[1]: Mounted /sysroot.526gitlab # [ 2.625318] systemd[1]: Reached target Initrd Root File System.527gitlab # [ 2.627887] systemd[1]: Starting Mountpoints Configured in the Real Root...528gitlab # [ 2.661610] systemd-sysroot-fstab-check[161]: /sysroot should be mounted in the initrd, will request daemon-reload.529gitlab # [ 2.663629] systemd[1]: Reload requested from client PID 161 ('systemd-sysroot') (unit initrd-parse-etc.service)...530gitlab # [ 2.666831] systemd[1]: Reloading...531gitlab # [ 2.766524] systemd[1]: Reloading finished in 103 ms.532gitlab # [ 2.817931] systemd-sysroot-fstab-check[161]: Requesting initrd-fs.target/start/replace...533gitlab # [ 2.819687] systemd-sysroot-fstab-check[161]: Requesting swap.target/start/replace...534gitlab # [ 2.823297] systemd[1]: initrd-parse-etc.service: Deactivated successfully.535gitlab # [ 2.824987] systemd[1]: Finished Mountpoints Configured in the Real Root.536gitlab # [ 2.826013] systemd[1]: initrd-parse-etc.service: Triggering OnSuccess= dependencies.537gitlab # [ 2.827112] systemd[1]: Starting Load Kernel Module 9pnet_virtio...538gitlab # [ 2.848529] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.539gitlab # [ 2.850534] systemd[1]: Finished Load Kernel Module 9pnet_virtio.540gitlab # [ 3.021374] (udev-worker)[136]: mtd0ro: Failed to find and pin callout binary "/nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/mtd_probe": No such file or directory541gitlab # [ 3.024496] (udev-worker)[136]: mtd0ro: /nix/store/x3ma60wd3w0db1m4dr1xln72q96lnm3b-systemd-261.1/lib/udev/rules.d/75-probe_mtd.rules:5 IMPORT{program}="mtd_probe $devnode": Failed to execute "mtd_probe /dev/mtd0ro", ignoring: No such file or directory542gitlab # [ 3.031936] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.543gitlab # [ 3.033309] systemd[1]: Stopped Virtual Console Setup.544gitlab # [ 3.034440] systemd[1]: Stopping Virtual Console Setup...545gitlab # [ 3.035268] systemd[1]: Starting Virtual Console Setup...546gitlab # [ 3.071055] systemd-vconsole-setup[183]: Configuration of first virtual console was skipped, ignoring remaining ones.547gitlab # [ 3.073309] systemd[1]: Finished Virtual Console Setup.548gitlab # [ 3.386409] systemd[1]: Mounting /sysroot/nix/.ro-store...549gitlab # [ 3.388276] systemd[1]: Mounting /sysroot/nix/.rw-store...550gitlab # [ 3.391832] systemd[1]: Mounting /sysroot/run...551gitlab # [ 3.393662] systemd[1]: Mounting /sysroot/tmp/shared...552gitlab # [ 3.406098] systemd[1]: Mounting /sysroot/tmp/xchg...553gitlab # [ 3.419721] systemd[1]: Mounted /sysroot/nix/.rw-store.554gitlab # [ 3.421178] systemd[1]: Mounted /sysroot/run.555gitlab # [ 3.423256] systemd[1]: Starting rw-sysroot-nix-store.service...556gitlab # [ 3.429441] systemd[1]: Mounted /sysroot/tmp/shared.557gitlab # [ 3.430752] systemd[1]: Mounted /sysroot/tmp/xchg.558gitlab # [ 3.438870] erofs (device vdb): mounted with root inode @ nid 128.559gitlab # [ 3.439415] systemd[1]: Mounted /sysroot/nix/.ro-store.560gitlab # [ 3.440821] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.561gitlab # [ 3.442461] systemd[1]: Finished rw-sysroot-nix-store.service.562gitlab # [ 4.385875] systemd[1]: Mounting /sysroot/nix/store...563gitlab # [ 4.427061] systemd[1]: Mounted /sysroot/nix/store.564gitlab # [ 4.428422] systemd[1]: Reached target Initrd File Systems.565gitlab # [ 4.429974] systemd[1]: Starting Find NixOS closure...566gitlab # [ 4.431870] systemd[1]: Starting Create Volatile Files and Directories in the Real Root...567gitlab # [ 4.461214] systemd[1]: Finished Create Volatile Files and Directories in the Real Root.568gitlab # [ 4.463190] systemd[1]: sysroot-run-credentials-systemd\x2dtmpfiles\x2dsetup\x2dsysroot.service.mount: Deactivated successfully.569gitlab # [ 4.474814] systemd[1]: Finished Find NixOS closure.570gitlab # [ 4.476184] systemd[1]: Reached target Initrd Default Target.571gitlab # [ 4.477482] systemd[1]: Starting Cleaning Up and Shutting Down Daemons...572gitlab # [ 4.497327] systemd[1]: Stopped target Initrd Default Target.573gitlab # [ 4.498732] systemd[1]: Stopped target Basic System.574gitlab # [ 4.499867] systemd[1]: Stopped target Initrd Root Device.575gitlab # [ 4.501222] systemd[1]: Stopped target Path Units.576gitlab # [ 4.502348] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.577gitlab # [ 4.504037] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.578gitlab # [ 4.505766] systemd[1]: Stopped target Slice Units.579gitlab # [ 4.507386] systemd[1]: Stopped target Socket Units.580gitlab # [ 4.513296] systemd[1]: Stopped target System Initialization.581gitlab # [ 4.514725] systemd[1]: Stopped target Swaps.582gitlab # [ 4.515760] systemd[1]: Stopped target Timer Units.583gitlab # [ 4.517018] systemd[1]: dbus.socket: Deactivated successfully.584gitlab # [ 4.518410] systemd[1]: Closed D-Bus System Message Bus Socket.585gitlab # [ 4.519749] systemd[1]: initrd-find-nixos-closure.service: Deactivated successfully.586gitlab # [ 4.521565] systemd[1]: Stopped Find NixOS closure.587gitlab # [ 4.522758] systemd[1]: Starting Load Kernel Module 9pnet_virtio...588gitlab # [ 4.524334] systemd[1]: Starting rw-sysroot-nix-store.service...589gitlab # [ 4.526051] systemd[1]: systemd-sysctl.service: Deactivated successfully.590gitlab # [ 4.527749] systemd[1]: Stopped Apply Kernel Variables.591gitlab # [ 4.529115] systemd[1]: systemd-modules-load.service: Deactivated successfully.592gitlab # [ 4.530764] systemd[1]: Stopped Load Kernel Modules.593gitlab # [ 4.532140] systemd[1]: systemd-tmpfiles-setup-sysroot.service: Deactivated successfully.594gitlab # [ 4.533968] systemd[1]: Stopped Create Volatile Files and Directories in the Real Root.595gitlab # [ 4.535798] systemd[1]: systemd-tmpfiles-setup.service: Deactivated successfully.596gitlab # [ 4.537552] systemd[1]: Stopped Create System Files and Directories.597gitlab # [ 4.539081] systemd[1]: Stopped target Local File Systems.598gitlab # [ 4.540369] systemd[1]: Stopped target Preparation for Local File Systems.599gitlab # [ 4.541895] systemd[1]: systemd-udev-trigger.service: Deactivated successfully.600gitlab # [ 4.543425] systemd[1]: Stopped Coldplug All udev Devices.601gitlab # [ 4.544708] systemd[1]: Stopping Rule-based Manager for Device Events and Files...602gitlab # [ 4.546332] systemd[1]: systemd-vconsole-setup.service: Deactivated successfully.603gitlab # [ 4.547864] systemd[1]: Stopped Virtual Console Setup.604gitlab # [ 4.549160] systemd[1]: initrd-cleanup.service: Deactivated successfully.605gitlab # [ 4.550526] systemd[1]: Finished Cleaning Up and Shutting Down Daemons.606gitlab # [ 4.551813] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.607gitlab # [ 4.553331] systemd[1]: Finished Load Kernel Module 9pnet_virtio.608gitlab # [ 4.554535] systemd[1]: rw-sysroot-nix-store.service: Deactivated successfully.609gitlab # [ 4.555860] systemd[1]: Finished rw-sysroot-nix-store.service.610gitlab # [ 4.557024] systemd[1]: systemd-udevd.service: Deactivated successfully.611gitlab # [ 4.558250] systemd[1]: Stopped Rule-based Manager for Device Events and Files.612gitlab # [ 4.559523] systemd[1]: systemd-udevd.service: Consumed 2.142s CPU time over 3.049s wall clock time, 31M memory peak.613gitlab # [ 4.561417] systemd[1]: systemd-udevd-control.socket: Deactivated successfully.614gitlab # [ 4.562691] systemd[1]: Closed udev Control Socket.615gitlab # [ 4.563568] systemd[1]: Starting Cleanup udev Database...616gitlab # [ 4.564571] systemd[1]: systemd-tmpfiles-setup-dev.service: Deactivated successfully.617gitlab # [ 4.565903] systemd[1]: Stopped Create Static Device Nodes in /dev.618gitlab # [ 4.566965] systemd[1]: systemd-tmpfiles-setup-dev-early.service: Deactivated successfully.619gitlab # [ 4.568369] systemd[1]: Stopped Create Static Device Nodes in /dev gracefully.620gitlab # [ 4.569573] systemd[1]: kmod-static-nodes.service: Deactivated successfully.621gitlab # [ 4.570762] systemd[1]: Stopped Create List of Static Device Nodes.622gitlab # [ 4.596234] systemd[1]: initrd-udevadm-cleanup-db.service: Deactivated successfully.623gitlab # [ 4.597978] systemd[1]: Finished Cleanup udev Database.624gitlab # [ 4.599199] systemd[1]: Reached target Switch Root.625gitlab # [ 4.600394] systemd[1]: Starting NixOS Activation...626gitlab # [ 4.705339] initrd-nixos-activation-start[221]: booting system configuration /nix/store/2hpmbb80a6jj3dmanfcy6yy2az91lf6i-nixos-system-gitlab-test627gitlab # [ 4.743028] initrd-nixos-activation-start[221]: running activation script...628gitlab # [ 4.972350] initrd-nixos-activation-start[247]: setting up /etc...629gitlab # [ 5.079380] systemd[1]: initrd-nixos-activation.service: Deactivated successfully.630gitlab # [ 5.081288] systemd[1]: Finished NixOS Activation.631gitlab # [ 5.082455] systemd[1]: Starting Switch Root...632gitlab # [ 5.112078] systemd[1]: Switching root.633gitlab # [ 5.265889] systemd-journald[94]: Received SIGTERM from PID 1 (systemd).634gitlab # [ 5.818797] systemd[1]: systemd 261.1 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)635gitlab # [ 5.818883] systemd[1]: Detected virtualization qemu.636gitlab # [ 5.818981] systemd[1]: Detected architecture arm64.637gitlab # [ 5.819086] systemd[1]: Detected first boot.638gitlab # [ 5.819827] systemd[1]: Initializing machine ID from random generator.639gitlab # [ 6.005482] systemd[1]: bpf-restrict-fs: LSM BPF program attached640gitlab # [ 6.087542] systemd[1]: Applying preset policy.641gitlab # [ 6.260196] systemd[1]: Populated /etc with preset unit settings.642gitlab # [ 6.405371] systemd[1]: initrd-switch-root.service: Deactivated successfully.643gitlab # [ 6.407119] systemd[1]: Stopped initrd-switch-root.service.644gitlab # [ 6.409106] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.645gitlab # [ 6.412314] systemd[1]: Created slice Slice /system/getty.646gitlab # [ 6.414467] systemd[1]: Created slice GitLab DevOps Platform Slice.647gitlab # [ 6.416740] systemd[1]: Created slice User and Session Slice.648gitlab # [ 6.417267] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.649gitlab # [ 6.417899] systemd[1]: Started Forward Password Requests to Wall Directory Watch.650gitlab # [ 6.418493] systemd[1]: Expecting device /dev/hvc0...651gitlab # [ 6.419835] systemd[1]: Expecting device /dev/ttyAMA0...652gitlab # [ 6.420374] systemd[1]: Reached target Local Encrypted Volumes.653gitlab # [ 6.420807] systemd[1]: Stopped target initrd-fs.target.654gitlab # [ 6.421183] systemd[1]: Stopped target initrd-root-fs.target.655gitlab # [ 6.421544] systemd[1]: Stopped target initrd-switch-root.target.656gitlab # [ 6.421913] systemd[1]: Reached target Virtual Machines and Containers.657gitlab # [ 6.422299] systemd[1]: Reached target Path Units.658gitlab # [ 6.422660] systemd[1]: Reached target Remote File Systems.659gitlab # [ 6.423198] systemd[1]: Reached target Slice Units.660gitlab # [ 6.423813] systemd[1]: Reached target Swaps.661gitlab # [ 6.426077] systemd[1]: Listening on Query the User Interactively for a Password.662gitlab # [ 6.428364] systemd[1]: Listening on Process Core Dump Socket.663gitlab # [ 6.430045] systemd[1]: Listening on Credential Encryption/Decryption.664gitlab # [ 6.431960] systemd[1]: Listening on Factory Reset Management.665gitlab # [ 6.432689] systemd[1]: Listening on Hostname Service Socket.666gitlab # [ 6.438255] systemd[1]: Starting Journal Log Access Socket...667gitlab # [ 6.439597] systemd[1]: Listening on Journal Audit Socket.668gitlab # [ 6.441084] systemd[1]: Listening on Console Output Muting Service Socket.669gitlab # [ 6.441534] systemd[1]: Listening on Userspace Out-Of-Memory (OOM) Killer Socket.670gitlab # [ 6.441812] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os671gitlab # [ 6.442045] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki672gitlab # [ 6.444313] systemd[1]: Listening on Disk Repartitioning Service Socket.673gitlab # [ 6.444822] systemd[1]: Listening on udev Control Socket.674gitlab # [ 6.445464] systemd[1]: Listening on udev Varlink Socket.675gitlab # [ 6.449626] systemd[1]: Mounting Huge Pages File System...676gitlab # [ 6.454107] systemd[1]: Mounting POSIX Message Queue File System...677gitlab # [ 6.460196] systemd[1]: Mounting Kernel Debug File System...678gitlab # [ 6.466014] systemd[1]: Mounting Kernel Trace File System...679gitlab # [ 6.476755] systemd[1]: Starting Create List of Static Device Nodes...680gitlab # [ 6.483127] systemd[1]: Starting Load Kernel Module 9pnet_virtio...681gitlab # [ 6.483835] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs682gitlab # [ 6.488814] systemd[1]: Mounting Kernel Configuration File System...683gitlab # [ 6.489444] systemd[1]: Load Kernel Module drm skipped, unmet condition check ConditionKernelModuleLoaded=!drm684gitlab # [ 6.490067] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore685gitlab # [ 6.496903] systemd[1]: Starting Load Kernel Module fuse...686gitlab # [ 6.497568] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67687gitlab # [ 6.501960] systemd[1]: Starting Journal Service...688gitlab # [ 6.508579] systemd[1]: Starting Load Kernel Modules...689gitlab # [ 6.519111] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...690gitlab # [ 6.527100] systemd[1]: Starting Remount Root and Kernel File Systems...691gitlab # [ 6.527621] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os692gitlab # [ 6.528610] fuse: init (API version 7.45)693gitlab # [ 6.532321] systemd[1]: Starting Coldplug All udev Devices...694gitlab # [ 6.538809] systemd[1]: Listening on Journal Log Access Socket.695gitlab # [ 6.539376] systemd-journald[319]: Collecting audit messages is enabled.696gitlab # [ 6.542393] systemd[1]: Mounted Huge Pages File System.697gitlab # [ 6.547093] systemd[1]: Mounted POSIX Message Queue File System.698gitlab # [ 6.547916] systemd[1]: Mounted Kernel Debug File System.699gitlab # [ 6.548684] systemd[1]: Mounted Kernel Trace File System.700gitlab # [ 6.549794] systemd[1]: Finished Create List of Static Device Nodes.701gitlab # [ 6.551706] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.702gitlab # [ 6.553886] systemd[1]: Finished Load Kernel Module 9pnet_virtio.703gitlab # [ 6.555101] systemd[1]: Mounted Kernel Configuration File System.704gitlab # [ 6.556075] systemd[1]: modprobe@fuse.service: Deactivated successfully.705gitlab # [ 6.557077] systemd[1]: Finished Load Kernel Module fuse.706gitlab # [ 6.558233] systemd[1]: Finished Load Kernel Modules.707gitlab # [ 6.564444] systemd[1]: Queued start job for default target Multi-User System.708gitlab # [ 6.567084] systemd[1]: Mounting FUSE Control File System...709gitlab # [ 6.567135] systemd[1]: Unnecessary job was removed for /dev/vdb.710gitlab # [ 6.568560] systemd[1]: systemd-journald.service: Deactivated successfully.[ 6.571223] systemd[1]: Starting Firewall...711gitlab # 712gitlab # [ 6.573679] EXT4-fs (vda): re-mounted 981d9538-fe2e-4bd9-ba3f-58cd8945b958.713gitlab # [ 6.573029] systemd-modules-load[320]: Using 3 probe threads714gitlab # [ 6.574299] systemd-modules-load[320]: Module 'atkbd' is built in715gitlab # [ 6.578089] systemd-modules-load[320]: Module 'loop' is built in716gitlab # [ 6.579302] systemd-modules-load[320]: Inserted module 'tls'[ 6.582314] systemd[1]: Starting Apply Kernel Variables...717gitlab # 718gitlab # [ 6.581993] systemd-oomd[322]: No swap; memory pressure usage will be degraded719gitlab # [ 6.591091] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...720gitlab # [ 6.594658] systemd[1]: Started Journal Service.721gitlab # [ 6.593876] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.722gitlab # [ 6.597682] systemd[1]: Finished Remount Root and Kernel File Systems.723gitlab # [ 6.598933] systemd[1]: Mounted FUSE Control File System.724gitlab # [ 6.604005] systemd[1]: Finished Apply Kernel Variables.725gitlab # [ 6.610795] systemd[1]: Listening on Disk Image Download Service Socket.726gitlab # [ 6.615553] systemd[1]: Starting Flush Journal to Persistent Storage...727gitlab # [ 6.616769] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore728gitlab # [ 6.620203] systemd[1]: Starting Load/Save OS Random Seed...729gitlab # [ 6.621225] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os730gitlab # [ 6.624264] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.731gitlab # [ 6.631151] systemd-journald[319]: Received client request to flush runtime journal.732gitlab # [ 6.653543] systemd[1]: Starting Create Static Device Nodes in /dev...733gitlab # [ 6.655255] systemd[1]: Finished Create Static Device Nodes in /dev.734gitlab # [ 6.656480] systemd[1]: Reached target Preparation for Local File Systems.735gitlab # [ 6.657523] systemd[1]: Starting Rule-based Manager for Device Events and Files...736gitlab # [ 6.659023] systemd[1]: Finished Flush Journal to Persistent Storage.737gitlab # [ 6.659963] systemd[1]: Finished Load/Save OS Random Seed.738gitlab # [ 6.660849] systemd[1]: Reached target First Boot Complete.739gitlab # [ 6.710862] systemd-udevd[361]: Using default interface naming scheme 'v261'.740gitlab # [ 6.743708] systemd[1]: Started Rule-based Manager for Device Events and Files.741gitlab # [ 6.918985] systemd[1]: Finished Firewall.742gitlab # [ 7.213685] systemd[1]: Finished Coldplug All udev Devices.743gitlab # [ 7.257283] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs744gitlab # [ 7.298397] systemd[1]: Condition check resulted in /dev/hvc0 being skipped.745gitlab # [ 7.322596] (udev-worker)[401]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.746gitlab # [ 7.326563] systemd[1]: Condition check resulted in /dev/ttyAMA0 being skipped.747gitlab # [ 7.327818] (udev-worker)[401]: Network interface NamePolicy= disabled on kernel command line.748gitlab # [ 7.335941] (udev-worker)[474]: Network interface NamePolicy= disabled on kernel command line.749gitlab # [ 7.338398] systemd[1]: Condition check resulted in Virtio network device being skipped.750gitlab # [ 7.340285] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore751gitlab # [ 7.342490] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc67752gitlab # [ 7.345393] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore753gitlab # [ 7.347610] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os754gitlab # [ 7.349010] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os755gitlab # [ 7.410635] mousedev: PS/2 mouse device common for all mice756gitlab # [ 7.412129] systemd[1]: Mounting /run/wrappers...757gitlab # [ 7.440117] systemd[1]: Mounted /run/wrappers.758gitlab # [ 7.441002] systemd[1]: Reached target Local File Systems.759gitlab # [ 7.441895] systemd[1]: Listening on Boot Loader Control Service Socket.760gitlab # [ 7.442946] systemd[1]: Starting register-nix-paths.service...761gitlab # [ 7.443849] systemd[1]: Starting Create SUID/SGID Wrappers...762gitlab # [ 7.449632] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.763gitlab # [ 7.463411] systemd[1]: Starting Save Transient machine-id to Disk...764gitlab # [ 7.464692] systemd[1]: Starting Create System Files and Directories...765gitlab # [ 7.494727] systemd[1]: etc-machine\x2did.mount: Deactivated successfully.766gitlab # [ 7.500401] systemd[1]: Finished Save Transient machine-id to Disk.767gitlab # [ 7.501472] systemd[1]: Finished Create System Files and Directories.768gitlab # [ 7.502946] systemd[1]: Starting Rebuild Journal Catalog...769gitlab # [ 7.508285] systemd[1]: Starting Record System Boot/Shutdown in UTMP...770gitlab # [ 7.540310] systemd[1]: Finished Record System Boot/Shutdown in UTMP.771gitlab # [ 7.562130] systemd[1]: Finished Rebuild Journal Catalog.772gitlab # [ 7.566012] systemd[1]: Starting Update is Completed...773gitlab # [ 7.597934] systemd[1]: Finished Update is Completed.774gitlab # [ 7.633735] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.775gitlab # [ 7.635987] systemd[1]: Finished Create SUID/SGID Wrappers.776gitlab # [ 7.636997] systemd[1]: suid-sgid-wrappers.service: Consumed 137ms CPU time over 190ms wall clock time, 6.1M memory peak, 1.9M read from disk.777gitlab # [ 7.871023] systemd[1]: Finished register-nix-paths.service.778gitlab # [ 7.872298] systemd[1]: Reached target System Initialization.779gitlab # [ 7.876032] input: QEMU Virtio Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:09.0/virtio8/input/input3780gitlab # [ 7.874867] systemd[1]: Started Discard unused filesystem blocks once a week.781gitlab # [ 7.878936] systemd[1]: Started logrotate.timer.782gitlab # [ 7.879747] systemd[1]: Started Daily Cleanup of Temporary Directories.783gitlab # [ 7.880979] systemd[1]: Reached target Timer Units.784gitlab # [ 7.882043] systemd[1]: Listening on D-Bus System Message Bus Socket.785gitlab # [ 7.883038] systemd[1]: Listening on Nix Daemon Socket.786gitlab # [ 7.897999] systemd[1]: Starting nixbot CI service socket...787gitlab # [ 7.907825] systemd[1]: Listening on Virtual Machine and Container Registration Service Socket.788gitlab # [ 7.931776] systemd[1]: Listening on nixbot CI service socket.789gitlab # [ 7.937090] systemd[1]: Reached target Socket Units.790gitlab # [ 7.951074] systemd[1]: Starting D-Bus System Message Bus...791gitlab # [ 7.996330] systemd[1]: Starting Virtual Console Setup...792gitlab # [ 8.001437] dbus-broker-launch[566]: Looking up NSS user entry for 'systemd-timesync'...793gitlab # [ 8.002958] dbus-broker-launch[566]: NSS returned no entry for 'systemd-timesync'794gitlab # [ 8.004275] dbus-broker-launch[566]: Invalid user-name in /nix/store/9zcyylhndlnizbwn48627991zr1jxg4p-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"795gitlab # [ 8.006618] systemd[1]: Started D-Bus System Message Bus.796gitlab # [ 8.016457] dbus-broker-launch[566]: Ready797gitlab # [ 8.017382] systemd[1]: Reached target Basic System.798gitlab # [ 8.019457] systemd[1]: Started backdoor.service.799gitlab # [ 8.022018] systemd[1]: Starting gitlab-config.service...800gitlab # [ 8.026043] systemd[1]: Starting Import lastlog data into lastlog2 database...801gitlab # [ 8.030561] systemd[1]: Starting Logrotate configuration check...802gitlab # [ 8.035749] systemd[1]: Starting Name Service Cache Daemon (nsncd)...803gitlab # [ 8.045730] systemd[1]: Starting Post-Boot Actions...804gitlab # connecting to host...805gitlab # [ 8.048918] systemd[1]: Started Reset console on configuration changes.806gitlab # [ 8.057577] logrotate[576]: warning: logrotate in debug mode does nothing except printing debug messages! Consider using verbose mode (-v) instead if this is not what you want.807gitlab # [ 8.063628] logrotate[576]: reading config file /nix/store/22afkmbkaxyx50bj9lbkkpmz7pd69hsf-logrotate.conf808gitlab # [ 8.065640] logrotate[576]: note: 'monthly' overrides previously specified 'weekly'809gitlab # [ 8.066878] logrotate[576]: Reading state from file: /var/lib/logrotate.status810gitlab # [ 8.067867] logrotate[576]: state file /var/lib/logrotate.status does not exist811gitlab: Guest shell says: b'Spawning backdoor root shell...\n'812gitlab # [ 8.069051] logrotate[576]: Allocating hash table for state file, size 64 entries813gitlab # [ 8.070390] logrotate[576]: Handling 4 logs814gitlab # [ 8.071140] logrotate[576]: rotating pattern: "/var/log/btmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed815gitlab: connected to guest root shell816gitlab # [ 8.073903] logrotate[576]: considering log /var/log/btmp817gitlab: (connecting took 8.53 seconds)818gitlab # [ 8.074595] logrotate[576]: Creating new state819gitlab: (finished: waiting for the VM to finish booting, in 8.53 seconds)820gitlab # [ 8.075172] logrotate[576]: Now: 2026-10-02 16:52821gitlab # [ 8.075824] logrotate[576]: Last rotated at 2026-10-02 16:00822gitlab # [ 8.076652] logrotate[576]: log does not need rotating (log has already been rotated)823gitlab # [ 8.077685] logrotate[576]: rotating pattern: "/var/log/wtmp" monthly empty log files are not rotated, only log files >= 1048576 bytes are rotated, (1 rotations), old logs are removed824gitlab # [ 8.079783] logrotate[576]: considering log /var/log/wtmp825gitlab # [ 8.081020] logrotate[576]: Creating new state826gitlab # [ 8.081611] logrotate[576]: Now: 2026-10-02 16:52827gitlab # [ 8.082359] logrotate[576]: Last rotated at 2026-10-02 16:00828gitlab # [ 8.083152] logrotate[576]: log does not need rotating (log has already been rotated)829gitlab # [ 8.084715] logrotate[576]: rotating pattern: "/var/gitlab/state/log/*.log" after 1 days empty log files are not rotated, (30 rotations), old logs are removed830gitlab # [ 8.086412] logrotate[576]: switching euid from 0 to 165 and egid from 0 to 165 (pid 576)831gitlab # [ 8.087449] logrotate[576]: considering log /var/gitlab/state/log/*.log832gitlab # [ 8.088302] logrotate[576]: log /var/gitlab/state/log/*.log does not exist -- skipping833gitlab # [ 8.089310] logrotate[576]: Creating new state834gitlab # [ 8.089921] logrotate[576]: switching euid from 165 to 0 and egid from 165 to 0 (pid 576)835gitlab # [ 8.091073] logrotate[576]: rotating pattern: "/var/log/nginx/*.log" weekly empty log files are not rotated, (26 rotations), old logs are removed836gitlab # [ 8.093358] logrotate[576]: switching euid from 0 to 60 and egid from 0 to 60 (pid 576)837gitlab # [ 8.094799] logrotate[576]: considering log /var/log/nginx/*.log838gitlab # [ 8.095610] logrotate[576]: log /var/log/nginx/*.log does not exist -- skipping839gitlab # [ 8.096691] logrotate[576]: Creating new state840gitlab # [ 8.097308] logrotate[576]: not running postrotate script, since no logs were rotated841gitlab # [ 8.098390] logrotate[576]: switching euid from 60 to 0 and egid from 60 to 0 (pid 576)842gitlab # [ 8.099509] systemd[1]: Starting resolvconf update...843gitlab # [ 8.104068] nsncd[577]: Oct 02 16:52:43.010 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"844gitlab # [ 8.106433] systemd[1]: Finished Logrotate configuration check.845gitlab # [ 8.107731] systemd[1]: Started Name Service Cache Daemon (nsncd).846gitlab # [ 8.109171] systemd[1]: Finished Post-Boot Actions.847gitlab # [ 8.110062] systemd[1]: Finished Import lastlog data into lastlog2 database.848gitlab # [ 8.111146] systemd[1]: Reached target Host and Network Name Lookups.849gitlab # [ 8.112189] systemd[1]: Reached target User and Group Name Lookups.850gitlab # [ 8.113184] systemd-vconsole-setup[570]: Configuration of first virtual console was skipped, ignoring remaining ones.851gitlab # [ 8.114780] systemd[1]: Starting User Login Management...852gitlab # [ 8.115656] systemd[1]: Finished Virtual Console Setup.853gitlab # [ 8.149951] systemd[1]: Stopped target Host and Network Name Lookups.854gitlab # [ 8.150990] systemd[1]: Stopping Host and Network Name Lookups...855gitlab # [ 8.152541] systemd[1]: Stopped target User and Group Name Lookups.856gitlab # [ 8.153524] systemd[1]: Stopping User and Group Name Lookups...857gitlab # [ 8.154467] systemd[1]: Stopping Name Service Cache Daemon (nsncd)...858gitlab # [ 8.155412] systemd[1]: nscd.service: Deactivated successfully.859gitlab # [ 8.156414] systemd[1]: Stopped Name Service Cache Daemon (nsncd).860gitlab # [ 8.157302] systemd[1]: nscd.service: Consumed 25ms CPU time over 118ms wall clock time, 7.6M memory peak, 5.8M read from disk.861gitlab # [ 8.160633] systemd-logind[611]: New seat seat0.862gitlab # [ 8.163156] systemd-logind[611]: Watching system buttons on /dev/input/event3 (QEMU Virtio Keyboard)863gitlab # [ 8.165348] systemd-logind[611]: Watching system buttons on /dev/input/event1 (QEMU QEMU USB Keyboard)864gitlab # [ 8.166762] systemd-logind[611]: Watching system buttons on /dev/input/event0 (gpio-keys)865gitlab # [ 8.167890] systemd[1]: Starting Name Service Cache Daemon (nsncd)...866gitlab # [ 8.169446] systemd[1]: Started User Login Management.867gitlab # [ 8.181996] systemd[1]: Starting linger-users.service...868gitlab # [ 8.197227] nsncd[666]: Oct 02 16:52:43.133 INFO started, config: Config { ignored_request_types: {}, worker_count: 8, handoff_timeout: 10s }, path: "/var/run/nscd/socket"869gitlab # [ 8.199380] systemd[1]: Started Name Service Cache Daemon (nsncd).870gitlab # [ 8.200677] systemd[1]: Finished resolvconf update.871gitlab # [ 8.201470] systemd[1]: linger-users.service: Deactivated successfully.872gitlab # [ 8.202493] systemd[1]: Finished linger-users.service.873gitlab # [ 8.203271] systemd[1]: Reached target Preparation for Network.874gitlab # [ 8.204129] systemd[1]: Reached target Host and Network Name Lookups.875gitlab # [ 8.205075] systemd[1]: Reached target User and Group Name Lookups.876gitlab # [ 8.207973] systemd[1]: Starting DHCP Client...877gitlab # [ 8.210825] systemd[1]: Starting Address configuration of eth1...878gitlab # [ 8.216315] systemd[1]: Starting Extra networking commands....879gitlab # [ 8.241807] network-addresses-eth1-start[696]: adding address 192.168.1.1/24... done880gitlab # [ 8.247201] network-addresses-eth1-start[696]: adding address 2001:db8:1::1/64... done881gitlab # [ 8.268978] systemd[1]: Finished Address configuration of eth1.882gitlab # [ 8.298455] systemd[1]: Finished Extra networking commands..883gitlab # [ 8.299551] systemd[1]: Reached target Network.884gitlab # [ 8.301980] systemd[1]: Starting gitlab-workhorse.service...885gitlab # [ 8.308091] systemd[1]: Starting Nginx Web Server...886gitlab # [ 8.309273] dhcpcd[707]: dhcpcd-10.3.2 starting887gitlab # [ 8.310935] dhcpcd[763]: dev: loaded udev888gitlab # [ 8.313709] systemd[1]: Starting PostgreSQL Server...889gitlab # [ 8.316393] systemd[1]: Starting Redis Server - redis-gitlab...890gitlab # [ 8.321842] systemd[1]: Starting Permit User Sessions...891gitlab # [ 8.329725] 8021q: 802.1Q VLAN Support v1.8892gitlab # [ 8.330147] 8021q: adding VLAN 0 to HW filter on device eth1893gitlab # [ 8.340806] systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.894gitlab # [ 8.356198] systemd[1]: Finished Permit User Sessions.895gitlab # [ 8.360594] systemd[1]: Started Getty on tty1.896gitlab # [ 8.361844] systemd[1]: Reached target Login Prompts.897gitlab # [ 8.388404] cfg80211: Loading compiled-in X.509 certificates for regulatory database898gitlab # [ 8.402403] Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'899gitlab # [ 8.403081] Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600'900gitlab # [ 8.404527] faux_driver regulatory: Direct firmware load for regulatory.db failed with error -2901gitlab # [ 8.404872] cfg80211: failed to load regulatory.db902gitlab # [ 8.443979] 8021q: adding VLAN 0 to HW filter on device eth0903gitlab # [ 8.442401] dhcpcd[763]: eth0: waiting for carrier904gitlab # [ 8.445134] dhcpcd[763]: eth0: carrier acquired905gitlab # [ 8.448912] dhcpcd[763]: DUID 00:01:00:01:32:52:9d:db:52:54:00:12:34:56906gitlab # [ 8.450061] dhcpcd[763]: eth0: IAID 00:12:34:56907gitlab # [ 8.450779] dhcpcd[763]: eth0: adding address fe80::5054:ff:fe12:3456908gitlab # [ 8.481679] nginx-pre-start[791]: nginx: the configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf syntax is ok909gitlab # [ 8.483538] nginx-pre-start[791]: nginx: configuration file /nix/store/ybdwy2xmslvjmdkxgrrhy1x5rwh45z0q-nginx.conf test is successful910gitlab # [ 8.491322] systemd[1]: Started Nginx Web Server.911gitlab # [ 8.508583] redis-server[786]: Supervised by systemd. Please make sure you set appropriate values for TimeoutStartSec and TimeoutStopSec in your service unit.912gitlab # [ 8.510784] redis-server[786]: oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo913gitlab # [ 8.512134] redis-server[786]: Redis version=8.8.1, bits=64, commit=00000000, modified=0, pid=786, just started914gitlab # [ 8.513923] redis-server[786]: Configuration loaded915gitlab # [ 8.514832] systemd[1]: Started Redis Server - redis-gitlab.916gitlab # [ 8.515763] redis[786]: monotonic clock: ARM CNTVCT @ 1000 ticks/us917gitlab # [ 8.517002] redis[786]: Running mode=standalone, port=0.918gitlab # [ 8.517915] redis[786]: Server initialized919gitlab # [ 8.518801] redis[786]: Ready to accept connections unix920gitlab # [ 8.520117] redis[786]: WARNING: Redis does not require authentication. Redis will accept connections from any local client.921gitlab # [ 8.535543] postgresql-pre-start[794]: The files belonging to this database system will be owned by user "postgres".922gitlab # [ 8.537322] postgresql-pre-start[794]: This user must also own the server process.923gitlab # [ 8.542658] postgresql-pre-start[794]: The database cluster will be initialized with locale "en_US.UTF-8".924gitlab # [ 8.544221] postgresql-pre-start[794]: The default database encoding has accordingly been set to "UTF8".925gitlab # [ 8.545499] postgresql-pre-start[794]: The default text search configuration will be set to "english".926gitlab # [ 8.546916] postgresql-pre-start[794]: Data page checksums are enabled.927gitlab # [ 8.547923] postgresql-pre-start[794]: fixing permissions on existing directory /var/lib/postgresql/18 ... ok928gitlab # [ 8.549426] postgresql-pre-start[794]: creating subdirectories ... ok929gitlab # [ 8.551308] postgresql-pre-start[794]: selecting dynamic shared memory implementation ... posix930gitlab # [ 8.586807] postgresql-pre-start[794]: selecting default "max_connections" ... 100931gitlab # [ 8.636119] postgresql-pre-start[794]: selecting default "shared_buffers" ... 128MB932gitlab # [ 8.942920] systemd[1]: Started gitlab-workhorse.service.933gitlab # [ 9.119550] postgresql-pre-start[794]: selecting default time zone ... UTC934gitlab # [ 9.122409] postgresql-pre-start[794]: creating configuration files ... ok935gitlab # [ 9.148627] gitlab-workhorse[822]: time="2026-10-02T16:52:44Z" level=info msg=Starting build_time=19700101.000000 version=19.0.4936gitlab # [ 9.150287] gitlab-workhorse[822]: time="2026-10-02T16:52:44Z" level=info msg="Using redis/go-redis"937gitlab # [ 9.151624] gitlab-workhorse[822]: time="2026-10-02T16:52:44Z" level=info msg="Running upstream server" address=/run/gitlab/gitlab-workhorse.socket network=unix938gitlab # [ 9.369472] postgresql-pre-start[794]: running bootstrap script ... ok939gitlab # [ 9.965726] dhcpcd[763]: eth0: soliciting an IPv6 router940gitlab # [ 9.967215] dhcpcd[763]: eth0: Router Advertisement from fe80::2941gitlab # [ 9.968659] dhcpcd[763]: eth0: adding address fec0::5054:ff:fe12:3456/64942gitlab # [ 9.970287] dhcpcd[763]: eth0: adding route to fec0::/64943gitlab # [ 9.971548] dhcpcd[763]: eth0: adding default route via fe80::2944gitlab # [ 9.998301] postgresql-pre-start[794]: performing post-bootstrap initialization ... ok945gitlab # [ 10.213077] postgresql-pre-start[794]: syncing data to disk ... ok946gitlab # [ 10.214672] postgresql-pre-start[794]: initdb: warning: enabling "trust" authentication for local connections947gitlab # [ 10.217636] postgresql-pre-start[794]: initdb: hint: You can change this by editing pg_hba.conf or using the option -A, or --auth-local and --auth-host, the next time you run initdb.948gitlab # [ 10.221974] postgresql-pre-start[794]: Success. You can now start the database server using:949gitlab # [ 10.223949] postgresql-pre-start[794]: pg_ctl -D /var/lib/postgresql/18 -l logfile start950gitlab # [ 10.381438] postgres[855]: [855] LOG: starting PostgreSQL 18.4 on aarch64-unknown-linux-gnu, compiled by clang version 21.1.8, 64-bit951gitlab # [ 10.384339] postgres[855]: [855] LOG: listening on IPv6 address "::1", port 5432952gitlab # [ 10.386269] postgres[855]: [855] LOG: listening on IPv4 address "127.0.0.1", port 5432953gitlab # [ 10.388130] postgres[855]: [855] LOG: listening on Unix socket "/run/postgresql/.s.PGSQL.5432"954gitlab # [ 10.390088] dhcpcd[763]: eth0: soliciting a DHCP lease955gitlab # [ 10.394166] postgres[866]: [866] LOG: database system was shut down at 2026-10-02 16:52:44 GMT956gitlab # [ 10.400753] dhcpcd[763]: eth0: offered 10.0.2.15 from 10.0.2.2957gitlab # [ 10.402511] postgres[855]: [855] LOG: database system is ready to accept connections958gitlab # [ 10.406393] systemd[1]: Started PostgreSQL Server.959gitlab # [ 10.411223] systemd[1]: Starting PostgreSQL Setup Scripts...960gitlab # [ 10.420380] dhcpcd[763]: eth0: probing address 10.0.2.15/24961gitlab # [ 10.646588] postgresql-setup-start[877]: CREATE DATABASE962gitlab # [ 10.682556] postgresql-setup-start[887]: CREATE ROLE963gitlab # [ 10.696745] postgresql-setup-start[889]: ALTER DATABASE964gitlab # [ 10.726741] postgresql-setup-start[894]: CREATE ROLE965gitlab # [ 10.729558] systemd[1]: Finished PostgreSQL Setup Scripts.966gitlab # [ 10.731073] systemd[1]: Reached target PostgreSQL.967gitlab # [ 10.734413] systemd[1]: Starting gitlab-postgresql.service...968gitlab # [ 10.739951] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[882]: mkdir -p /var/gitlab/state/home/.ssh: OK969gitlab # [ 10.742215] kmklqy2bdnfisygngf3cnp8jwzlh0j09-gitlab-config[882]: chmod 700 /var/gitlab/state/home/.ssh: OK970gitlab # [ 10.797383] systemd[1]: Finished gitlab-config.service.971gitlab # [ 10.800637] systemd[1]: Started gitaly.service.972gitlab # [ 10.804534] systemd[1]: Starting gitlab-db-config.service...973gitlab # [ 10.868000] gitlab-postgresql-start[911]: CREATE DATABASE974gitlab # [ 10.921977] gitlab-postgresql-start[923]: CREATE EXTENSION975gitlab # [ 11.012839] gitlab-postgresql-start[925]: CREATE EXTENSION976gitlab # [ 11.018564] systemd[1]: Finished gitlab-postgresql.service.977gitlab # [ 11.326692] gitaly[916]: time="2026-10-02T16:52:46.262Z" level=warning msg="The current configurations will cause Gitaly to fail to start up in future versions. Please run 'gitaly configuration validate < /nix/store/0r0qd84zrxhqq373hk9spv8j9pg0qlnq-gitaly.toml' and fix the errors that are printed." pid=916978gitlab # [ 11.332529] gitaly[916]: time="2026-10-02T16:52:46.263Z" level=info msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined" pid=916979gitlab # [ 11.336278] gitaly[916]: time="2026-10-02T16:52:46.263Z" level=info msg="grpc prometheus histograms enabled" latencies="[0.001 0.005 0.025 0.1 0.5 1 10 30 60 300 1500]" pid=916980gitlab # [ 11.341422] gitaly[916]: time="2026-10-02T16:52:46.263Z" level=error msg="error initializing distributed tracing" error="tracing is disabled" pid=916981gitlab # [ 11.344202] gitaly[916]: time="2026-10-02T16:52:46.263Z" level=info msg="Starting Gitaly" pid=916 version=982gitlab # [ 11.553918] gitaly[916]: time="2026-10-02T16:52:46.490Z" level=info msg="finished unpacking auxiliary binaries" duration_ms=226 pid=916983gitlab # [ 11.555847] gitaly[916]: time="2026-10-02T16:52:46.490Z" level=info msg="finished initializing bootstrap" duration_ms=0 pid=916984gitlab # [ 11.557962] gitaly[916]: time="2026-10-02T16:52:46.491Z" level=info msg="finished initializing command factory" duration_ms=0 pid=916985gitlab # [ 11.559905] gitaly[916]: time="2026-10-02T16:52:46.491Z" level=info msg="using Git binary" binary_path=/run/gitaly/gitaly-916/git-exec-778307810.d/git pid=916986gitlab # [ 11.562650] gitaly[916]: time="2026-10-02T16:52:46.496Z" level=info msg="finished detecting git version" duration_ms=5 pid=916987gitlab # [ 11.564696] gitaly[916]: time="2026-10-02T16:52:46.496Z" level=info msg="using Git version" pid=916 version=2.53.g988gitlab # [ 11.566459] gitaly[916]: time="2026-10-02T16:52:46.496Z" level=info msg="clearing disk cache object folder" pid=916 storage=default989gitlab # [ 11.568548] gitaly[916]: time="2026-10-02T16:52:46.497Z" level=info msg="moving disk cache object folder" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2517171939 pid=916 storage=default990gitlab # [ 11.571795] gitaly[916]: time="2026-10-02T16:52:46.497Z" level=info msg="disk cache object folder doesn't exist, no need to remove" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2517171939 pid=916 storage=default991gitlab # [ 11.575632] gitaly[916]: time="2026-10-02T16:52:46.497Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/cache path=/var/gitlab/state/repositories/+gitaly/cache pid=916992gitlab # [ 11.578588] gitaly[916]: time="2026-10-02T16:52:46.497Z" level=info msg="Starting file walker" disk_cache_path=/var/gitlab/state/repositories/+gitaly/state path=/var/gitlab/state/repositories/+gitaly/state pid=916993gitlab # [ 11.581691] gitaly[916]: time="2026-10-02T16:52:46.497Z" level=info msg="cleared all cache object files" cache_directory=/var/gitlab/state/repositories/+gitaly/tmp/diskcache2517171939 clear_duration_ms=0 pid=916 storage=default994gitlab # [ 11.585204] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=warning msg="Unable to set SO_REUSEPORT: zero downtime upgrades will not work" error="operation not supported" pid=916995gitlab # [ 11.587976] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=info msg="listening at address" address=/var/gitlab/state/tmp/sockets/gitaly.socket pid=916 schema=unix996gitlab # [ 11.590557] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=info msg="listening at address" address=/run/gitaly/gitaly-916/sock.d/intern pid=916 schema=unix997gitlab # [ 11.593069] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=info msg="finished tempdir cleaner walk" pid=916 storage=default time_ms=0998gitlab # [ 11.595370] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=info msg="counting repositories" pid=916999gitlab # [ 11.596980] gitaly[916]: time="2026-10-02T16:52:46.511Z" level=info msg="starting to count repositories" pid=916 storage_path=/var/gitlab/state/repositories1000gitlab # [ 11.599475] gitaly[916]: time="2026-10-02T16:52:46.512Z" level=info msg="completed counting repositories" counting_duration_ms=0 pid=916 storage_path=/var/gitlab/state/repositories1001gitlab # [ 11.602322] gitaly[916]: time="2026-10-02T16:52:46.512Z" level=info msg="completed counting all repositories" counting_duration=0 pid=9161002gitlab # [ 11.604464] gitaly[916]: time="2026-10-02T16:52:46.513Z" level=info msg="starting prometheus listener" address="localhost:9236" pid=9161003gitlab # [ 11.606522] gitaly[916]: time="2026-10-02T16:52:46.513Z" level=info msg="Started Gitaly" duration_ms=250 pid=9161004gitlab # [ 11.608153] gitaly[916]: time="2026-10-02T16:52:46.513Z" level=info msg="Gitaly graceful stop timeout duration" duration_s=1m0s pid=9161005gitlab # [ 11.610217] gitaly[916]: time="2026-10-02T16:52:46.514Z" level=info msg="maintenance: daily scheduled" pid=916 scheduled="2026-10-03 12:00:00 +0000 UTC"1006gitlab # [ 11.899755] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[919]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1007gitlab # [ 11.904865] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[919]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1008gitlab # [ 11.912438] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[919]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1009gitlab # [ 14.989386] gitaly[916]: time="2026-10-02T16:52:49.925Z" level=info msg="License database preloaded" duration_ms=3662 pid=9161010gitlab # [ 15.202888] dhcpcd[763]: eth0: leased 10.0.2.15 for 86400 seconds1011gitlab # [ 15.204424] dhcpcd[763]: eth0: adding route to 10.0.2.0/241012gitlab # [ 15.205665] dhcpcd[763]: eth0: adding default route via 10.0.2.21013gitlab # [ 15.311598] systemd[1]: Started DHCP Client.1014gitlab # [ 15.312996] systemd[1]: Reached target Network is Online.1015gitlab # [ 58.166023] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[919]: W, [2026-10-02T16:53:33.102011 #919] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1016gitlab # [ 59.216825] gitlab-workhorse[822]: time="2026-10-02T16:53:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1017gitlab # [ 61.336299] postgres[1458]: [1458] ERROR: relation "feature_gates" does not exist at character 5231018gitlab # [ 61.336549] postgres[1458]: [1458] STATEMENT: SELECT a.attname, format_type(a.atttypid, a.atttypmod),1019gitlab # [ 61.336617] postgres[1458]: pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod,1020gitlab # [ 61.336679] postgres[1458]: c.collname, col_description(a.attrelid, a.attnum) AS comment,1021gitlab # [ 61.336740] postgres[1458]: attidentity AS identity,1022gitlab # [ 61.336801] postgres[1458]: attgenerated as attgenerated1023gitlab # [ 61.336873] postgres[1458]: FROM pg_attribute a1024gitlab # [ 61.336937] postgres[1458]: LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum1025gitlab # [ 61.337014] postgres[1458]: LEFT JOIN pg_type t ON a.atttypid = t.oid1026gitlab # [ 61.337077] postgres[1458]: LEFT JOIN pg_collation c ON a.attcollation = c.oid AND a.attcollation <> t.typcollation1027gitlab # [ 61.337139] postgres[1458]: WHERE a.attrelid = '"feature_gates"'::regclass1028gitlab # [ 61.337203] postgres[1458]: AND a.attnum > 0 AND NOT a.attisdropped1029gitlab # [ 61.337264] postgres[1458]: ORDER BY a.attnum1030gitlab # [ 61.337350] postgres[1458]:1031gitlab # [ 66.637391] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1509]: psql:/var/gitlab/state/db/structure.sql:9: NOTICE: extension "btree_gist" already exists, skipping1032gitlab # [ 66.637646] b834zvbgv5v11acriba6350sglax02bw-gitlab-db-config[1509]: psql:/var/gitlab/state/db/structure.sql:11: NOTICE: extension "pg_trgm" already exists, skipping1033gitlab # [ 69.218729] gitlab-workhorse[822]: time="2026-10-02T16:53:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1034gitlab # [ 79.225646] gitlab-workhorse[822]: time="2026-10-02T16:53:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1035gitlab # [ 88.444446] postgres[1621]: [1621] ERROR: relation "table_without_models" does not exist1036gitlab # [ 88.444725] postgres[1621]: [1621] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1037gitlab # [ 89.226500] gitlab-workhorse[822]: time="2026-10-02T16:54:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1038gitlab # [ 99.233698] gitlab-workhorse[822]: time="2026-10-02T16:54:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1039gitlab # [ 105.958855] systemd[1]: Finished gitlab-db-config.service.1040gitlab # [ 105.966618] systemd[1]: Started gitlab-sidekiq.service.1041gitlab # [ 105.973278] systemd[1]: Starting gitlab.service...1042gitlab # [ 106.635576] bundle[1860]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1043gitlab # [ 106.635868] bundle[1860]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1044gitlab # [ 106.637872] bundle[1860]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1045gitlab # [ 109.241665] gitlab-workhorse[822]: time="2026-10-02T16:54:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1046gitlab # [ 111.452117] sidekiq-cluster[1859]: {"severity":"INFO","time":"2026-10-02T16:54:26.346Z","message":"Starting cluster with 1 processes"}1047gitlab # [ 111.536986] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.441Z","pid":1860,"message":"* Enabling systemd notification integration"}1048gitlab # [ 111.539544] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"Puma starting in cluster mode..."}1049gitlab # [ 111.541974] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Puma version: 8.0.1 (\"Into the Arena\")"}1050gitlab # [ 111.544482] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Ruby version: ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1051gitlab # [ 111.547657] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Min threads: 0"}1052gitlab # [ 111.549942] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Max threads: 4"}1053gitlab # [ 111.552150] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Environment: production"}1054gitlab # [ 111.554328] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Master PID: 1860"}1055gitlab # [ 111.556543] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Workers: 2"}1056gitlab # [ 111.558610] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Restarts: (✔) hot (✖) phased (✖) refork"}1057gitlab # [ 111.561543] bundle[1860]: {"timestamp":"2026-10-02T16:54:26.473Z","pid":1860,"message":"* Preloading application"}1058gitlab # [ 119.242589] gitlab-workhorse[822]: time="2026-10-02T16:54:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1059gitlab # [ 129.245133] gitlab-workhorse[822]: time="2026-10-02T16:54:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1060gitlab # [ 139.247750] gitlab-workhorse[822]: time="2026-10-02T16:54:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1061gitlab # [ 149.248776] gitlab-workhorse[822]: time="2026-10-02T16:55:04Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1062gitlab # [ 150.926148] bundle[1860]: W, [2026-10-02T16:55:05.862264 #1860] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1063gitlab # [ 155.144501] sidekiq-cluster[1909]: 2026-10-02T16:55:10.080Z pid=1909 tid=wp9 INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>1909, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>424222720}1064gitlab # [ 155.609230] sidekiq-cluster[1909]: W, [2026-10-02T16:55:10.545812 #1909] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1065gitlab # [ 155.680691] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:10.617Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1066gitlab # [ 158.895743] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.831Z","message":"GitLab reliable fetch activated!"}1067gitlab # [ 158.898687] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.831Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1068gitlab # [ 158.929851] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.866Z","message":"Cron Jobs - added job with name bbo_users_delete_unconfirmed_secondary in the namespace default"}1069gitlab # [ 158.933550] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.868Z","message":"Cron Jobs - added job with name background_operation_environments_auto_delete in the namespace default"}1070gitlab # [ 158.937346] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.871Z","message":"Cron Jobs - added job with name adjourned_group_deletion_worker in the namespace default"}1071gitlab # [ 158.940824] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.873Z","message":"Cron Jobs - added job with name authn_data_retention_authentication_event_archive_worker in the namespace default"}1072gitlab # [ 158.944774] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.876Z","message":"Cron Jobs - added job with name delete_expired_trigger_token_worker in the namespace default"}1073gitlab # [ 158.948267] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.878Z","message":"Cron Jobs - added job with name adjourned_projects_deletion_cron_worker in the namespace default"}1074gitlab # [ 158.951790] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.880Z","message":"Cron Jobs - added job with name stuck_ci_jobs_worker in the namespace default"}1075gitlab # [ 158.955231] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.882Z","message":"Cron Jobs - added job with name drop_timed_out_worker in the namespace default"}1076gitlab # [ 158.958617] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.884Z","message":"Cron Jobs - added job with name pipeline_schedule_worker in the namespace default"}1077gitlab # [ 158.962017] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.885Z","message":"Cron Jobs - added job with name ci_schedule_bulk_delete_job_artifact_cron_worker in the namespace default"}1078gitlab # [ 158.965796] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.887Z","message":"Cron Jobs - added job with name update_locked_unknown_artifacts_worker in the namespace default"}1079gitlab # [ 158.969436] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.890Z","message":"Cron Jobs - added job with name ci_partitioning_worker in the namespace default"}1080gitlab # [ 158.972690] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.892Z","message":"Cron Jobs - added job with name ci_pipelines_expire_artifacts_worker in the namespace default"}1081gitlab # [ 158.976119] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.894Z","message":"Cron Jobs - added job with name ci_schedule_delete_objects_worker in the namespace default"}1082gitlab # [ 158.979505] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.896Z","message":"Cron Jobs - added job with name environments_auto_stop_cron_worker in the namespace default"}1083gitlab # [ 158.983033] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.898Z","message":"Cron Jobs - added job with name repository_check_worker in the namespace default"}1084gitlab # [ 158.986214] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.900Z","message":"Cron Jobs - added job with name admin_email_worker in the namespace default"}1085gitlab # [ 158.989182] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.903Z","message":"Cron Jobs - added job with name personal_access_tokens_expiring_worker in the namespace default"}1086gitlab # [ 158.992274] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.905Z","message":"Cron Jobs - added job with name deploy_tokens_expiring_worker in the namespace default"}1087gitlab # [ 158.995113] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.908Z","message":"Cron Jobs - added job with name personal_access_tokens_expired_notification_worker in the namespace default"}1088gitlab # [ 158.998326] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.911Z","message":"Cron Jobs - added job with name resource_access_tokens_inactive_tokens_deletion_cron_worker in the namespace default"}1089gitlab # [ 159.001505] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.913Z","message":"Cron Jobs - added job with name repository_archive_cache_worker in the namespace default"}1090gitlab # [ 159.004141] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.916Z","message":"Cron Jobs - added job with name import_export_project_cleanup_worker in the namespace default"}1091gitlab # [ 159.006768] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.918Z","message":"Cron Jobs - added job with name gitlab_import_import_file_cleanup_worker in the namespace default"}1092gitlab # [ 159.009542] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.920Z","message":"Cron Jobs - added job with name ci_archive_traces_cron_worker in the namespace default"}1093gitlab # [ 159.011960] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.923Z","message":"Cron Jobs - added job with name members_expiring_worker in the namespace default"}1094gitlab # [ 159.014405] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.926Z","message":"Cron Jobs - added job with name remove_expired_members_worker in the namespace default"}1095gitlab # [ 159.016811] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.928Z","message":"Cron Jobs - added job with name remove_expired_group_links_worker in the namespace default"}1096gitlab # [ 159.019109] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.931Z","message":"Cron Jobs - added job with name remove_unaccepted_member_invites_worker in the namespace default"}1097gitlab # [ 159.021582] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.934Z","message":"Cron Jobs - added job with name prune_old_events_worker in the namespace default"}1098gitlab # [ 159.023803] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.936Z","message":"Cron Jobs - added job with name gitlab_export_prune_project_export_jobs_worker in the namespace default"}1099gitlab # [ 159.026379] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.939Z","message":"Cron Jobs - added job with name trending_projects_worker in the namespace default"}1100gitlab # [ 159.028561] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.941Z","message":"Cron Jobs - added job with name remove_unreferenced_lfs_objects_worker in the namespace default"}1101gitlab # [ 159.030774] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.944Z","message":"Cron Jobs - added job with name bulk_imports_stale_import_worker in the namespace default"}1102gitlab # [ 159.032968] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.946Z","message":"Cron Jobs - added job with name import_stuck_project_import_jobs in the namespace default"}1103gitlab # [ 159.035078] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.948Z","message":"Cron Jobs - added job with name jira_import_stuck_jira_import_jobs in the namespace default"}1104gitlab # [ 159.037321] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.950Z","message":"Cron Jobs - added job with name stuck_export_jobs_worker in the namespace default"}1105gitlab # [ 159.039371] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.953Z","message":"Cron Jobs - added job with name gitlab_service_ping_worker in the namespace default"}1106gitlab # [ 159.041631] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.955Z","message":"Cron Jobs - added job with name stuck_merge_jobs_worker in the namespace default"}1107gitlab # [ 159.043699] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.956Z","message":"Cron Jobs - added job with name pages_domain_verification_cron_worker in the namespace default"}1108gitlab # [ 159.046053] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.959Z","message":"Cron Jobs - added job with name pages_domain_removal_cron_worker in the namespace default"}1109gitlab # [ 159.048277] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.960Z","message":"Cron Jobs - added job with name pages_domain_ssl_renewal_cron_worker in the namespace default"}1110gitlab # [ 159.050495] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.963Z","message":"Cron Jobs - added job with name issue_due_scheduler_worker in the namespace default"}1111gitlab # [ 159.052654] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.965Z","message":"Cron Jobs - added job with name schedule_migrate_external_diffs_worker in the namespace default"}1112gitlab # [ 159.054868] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.968Z","message":"Cron Jobs - added job with name namespaces_prune_aggregation_schedules_worker in the namespace default"}1113gitlab # [ 159.057311] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.969Z","message":"Cron Jobs - added job with name container_expiration_policy_worker in the namespace default"}1114gitlab # [ 159.059530] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.971Z","message":"Cron Jobs - added job with name cleanup_container_registry_worker in the namespace default"}1115gitlab # [ 159.061869] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.973Z","message":"Cron Jobs - added job with name image_ttl_group_policy_worker in the namespace default"}1116gitlab # [ 159.064035] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.976Z","message":"Cron Jobs - added job with name cleanup_dependency_proxy_worker in the namespace default"}1117gitlab # [ 159.066211] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.978Z","message":"Cron Jobs - added job with name cleanup_package_registry_worker in the namespace default"}1118gitlab # [ 159.068416] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.981Z","message":"Cron Jobs - added job with name x509_issuer_crl_check_worker in the namespace default"}1119gitlab # [ 159.070552] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.983Z","message":"Cron Jobs - added job with name users_create_statistics_worker in the namespace default"}1120gitlab # [ 159.072816] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.995Z","message":"Cron Jobs - added job with name authorized_project_update_periodic_recalculate_worker in the namespace default"}1121gitlab # [ 159.075195] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:13.998Z","message":"Cron Jobs - added job with name update_container_registry_info_worker in the namespace default"}1122gitlab # [ 159.077530] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.000Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_manager in the namespace default"}1123gitlab # [ 159.079741] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.003Z","message":"Cron Jobs - added job with name postgres_dynamic_partitions_dropper in the namespace default"}1124gitlab # [ 159.082154] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.006Z","message":"Cron Jobs - added job with name analytics_usage_trends_count_job_trigger_worker in the namespace default"}1125gitlab # [ 159.145405] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.080Z","message":"Cron Jobs - added job with name member_invitation_reminder_emails_worker in the namespace default"}1126gitlab # [ 159.149078] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.084Z","message":"Cron Jobs - added job with name schedule_merge_request_cleanup_refs_worker in the namespace default"}1127gitlab # [ 159.152732] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.086Z","message":"Cron Jobs - added job with name manage_evidence_worker in the namespace default"}1128gitlab # [ 159.155950] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.088Z","message":"Cron Jobs - added job with name publish_release_worker in the namespace default"}1129gitlab # [ 159.159391] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.091Z","message":"Cron Jobs - added job with name user_status_cleanup_batch_worker in the namespace default"}1130gitlab # [ 159.163090] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.094Z","message":"Cron Jobs - added job with name ssh_keys_expired_notification_worker in the namespace default"}1131gitlab # [ 159.170539] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.098Z","message":"Cron Jobs - added job with name ssh_keys_expiring_soon_notification_worker in the namespace default"}1132gitlab # [ 159.174335] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.102Z","message":"Cron Jobs - added job with name users_deactivate_dormant_users_worker in the namespace default"}1133gitlab # [ 159.178045] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.107Z","message":"Cron Jobs - added job with name ci_delete_unit_tests_worker in the namespace default"}1134gitlab # [ 159.181526] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.109Z","message":"Cron Jobs - added job with name batched_background_migrations_worker in the namespace default"}1135gitlab # [ 159.185088] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.112Z","message":"Cron Jobs - added job with name batched_background_migration_worker_ci_database in the namespace default"}1136gitlab # [ 159.188799] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.118Z","message":"Cron Jobs - added job with name batched_background_migration_worker_sec_database in the namespace default"}1137gitlab # [ 159.193190] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.121Z","message":"Cron Jobs - added job with name background_operations_worker_main_database in the namespace default"}1138gitlab # [ 159.196807] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.122Z","message":"Cron Jobs - added job with name background_operations_worker_ci_database in the namespace default"}1139gitlab # [ 159.200115] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.124Z","message":"Cron Jobs - added job with name background_operations_worker_sec_database in the namespace default"}1140gitlab # [ 159.203423] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.127Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_main_database in the namespace default"}1141gitlab # [ 159.207002] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.129Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_ci_database in the namespace default"}1142gitlab # [ 159.210543] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.132Z","message":"Cron Jobs - added job with name background_operations_worker_cell_local_sec_database in the namespace default"}1143gitlab # [ 159.214336] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.134Z","message":"Cron Jobs - added job with name issues_reschedule_stuck_issue_rebalances in the namespace default"}1144gitlab # [ 159.220309] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.137Z","message":"Cron Jobs - added job with name projects_schedule_refresh_build_artifacts_size_statistics_worker in the namespace default"}1145gitlab # [ 159.223644] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.139Z","message":"Cron Jobs - added job with name inactive_projects_deletion_cron_worker in the namespace default"}1146gitlab # [ 159.226784] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.141Z","message":"Cron Jobs - added job with name loose_foreign_keys_cleanup_worker in the namespace default"}1147gitlab # [ 159.229649] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.144Z","message":"Cron Jobs - added job with name loose_foreign_keys_ci_pipelines_builds_cleanup_worker in the namespace default"}1148gitlab # [ 159.232742] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.146Z","message":"Cron Jobs - added job with name loose_foreign_keys_merge_request_diff_commit_cleanup_worker in the namespace default"}1149gitlab # [ 159.235739] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.149Z","message":"Cron Jobs - added job with name batched_git_ref_updates_cleanup_scheduler_worker in the namespace default"}1150gitlab # [ 159.238785] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.151Z","message":"Cron Jobs - added job with name ci_runner_versions_reconciliation_worker in the namespace default"}1151gitlab # [ 159.241667] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.154Z","message":"Cron Jobs - added job with name users_migrate_records_to_ghost_user_in_batches_worker in the namespace default"}1152gitlab # [ 159.244636] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.156Z","message":"Cron Jobs - added job with name ci_runners_stale_machines_cleanup_worker in the namespace default"}1153gitlab # [ 159.247370] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.159Z","message":"Cron Jobs - added job with name packages_cleanup_delete_orphaned_dependencies_worker in the namespace default"}1154gitlab # [ 159.250439] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.162Z","message":"Cron Jobs - added job with name cleanup_dangling_debian_package_files_worker in the namespace default"}1155gitlab # [ 159.253346] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.165Z","message":"Cron Jobs - added job with name object_storage_delete_stale_direct_uploads_worker in the namespace default"}1156gitlab # [ 159.256151] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.166Z","message":"Cron Jobs - added job with name service_desk_custom_email_verification_cleanup in the namespace default"}1157gitlab # [ 159.258826] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.169Z","message":"Cron Jobs - added job with name deactivated_pages_deployments_delete_cron_worker in the namespace default"}1158gitlab # [ 159.261607] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.171Z","message":"Cron Jobs - added job with name ci_schedule_unlock_pipelines_in_queue_worker in the namespace default"}1159gitlab # [ 159.264119] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.172Z","message":"Cron Jobs - added job with name ci_catalog_resources_process_sync_events_worker in the namespace default"}1160gitlab # [ 159.266588] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.175Z","message":"Cron Jobs - added job with name namespaces_process_outdated_namespace_descendants_cron_worker in the namespace default"}1161gitlab # [ 159.269287] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.176Z","message":"Cron Jobs - added job with name performance_bar_stats in the namespace default"}1162gitlab # [ 159.271366] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.180Z","message":"Cron Jobs - added job with name ci_catalog_resources_aggregate_last30_day_usage_worker in the namespace default"}1163gitlab # [ 159.274038] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.183Z","message":"Cron Jobs - added job with name ci_catalog_resources_cleanup_last_usages_worker in the namespace default"}1164gitlab # [ 159.276494] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.185Z","message":"Cron Jobs - added job with name ci_click_house_finished_pipelines_sync_worker in the namespace default"}1165gitlab # [ 159.278820] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.188Z","message":"Cron Jobs - added job with name deactivate_expired_deployments_cron_worker in the namespace default"}1166gitlab # [ 159.281292] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.203Z","message":"Cron Jobs - added job with name database_monitor_locked_tables_cron_worker in the namespace default"}1167gitlab # [ 159.283589] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.206Z","message":"Cron Jobs - added job with name merge_requests_process_scheduled_merge in the namespace default"}1168gitlab # [ 159.285978] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.208Z","message":"Cron Jobs - added job with name ci_schedule_old_pipelines_removal_cron_worker in the namespace default"}1169gitlab # [ 159.288330] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.212Z","message":"Cron Jobs - added job with name version_version_check_cron in the namespace default"}1170gitlab # [ 159.290520] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.215Z","message":"Cron Jobs - added job with name import_placeholder_user_cleanup_worker in the namespace default"}1171gitlab # [ 159.292811] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.217Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_grant_archive_worker in the namespace default"}1172gitlab # [ 159.295648] gitlab-workhorse[822]: time="2026-10-02T16:55:14Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1173gitlab # [ 159.298852] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.220Z","message":"Cron Jobs - added job with name authn_data_retention_oauth_access_token_archive_worker in the namespace default"}1174gitlab # [ 159.301413] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.222Z","message":"Cron Jobs - added job with name lost_transaction_recovery_worker in the namespace default"}1175gitlab # [ 159.303590] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.225Z","message":"Cron Jobs - added job with name topology_service_stale_requests_cleanup_worker in the namespace default"}1176gitlab # [ 159.306100] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.228Z","message":"Cron Jobs - added job with name cells_schedule_claims_verification_worker in the namespace default"}1177gitlab # [ 159.308395] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:14.230Z","message":"Cron Jobs - added job with name concurrency_limit_resume_worker in the namespace default"}1178gitlab # [ 169.265463] gitlab-workhorse[822]: time="2026-10-02T16:55:24Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1179gitlab # [ 179.276263] gitlab-workhorse[822]: time="2026-10-02T16:55:34Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1180gitlab # [ 189.277132] gitlab-workhorse[822]: time="2026-10-02T16:55:44Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1181gitlab # [ 196.189200] postgres[2664]: [2664] ERROR: relation "table_without_models" does not exist1182gitlab # [ 196.191167] postgres[2664]: [2664] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1183gitlab # [ 199.285519] gitlab-workhorse[822]: time="2026-10-02T16:55:54Z" level=error correlation_id= duration_ms=0 error="badgateway: failed to receive response: dial unix /var/gitlab/state/tmp/sockets/gitlab.socket: connect: no such file or directory" method=GET uri=1184gitlab # [ 201.097358] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:56.033Z","message":"Booted Rails 7.2.3.1 application in production environment"}1185gitlab # [ 201.104222] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:56.040Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1186gitlab # [ 201.107440] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:56.040Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1187gitlab # [ 201.110414] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:56.040Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1188gitlab # [ 201.113765] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:56.045Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1189gitlab # [ 201.149440] postgres[2711]: [2711] ERROR: relation "table_without_models" does not exist1190gitlab # [ 201.151228] postgres[2711]: [2711] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1191gitlab # [ 203.875175] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:58.811Z","message":"Cleaning working queues"}1192gitlab # [ 203.876969] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:58.812Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"041c6ce431c75be173757456","created_at":"2026-10-02T16:54:20.286Z","trace_propagation_headers":{"sentry-trace":"815d05d9ebf949aaa42b5e9558def271-416cca0e315142ec","baggage":"sentry-trace_id=815d05d9ebf949aaa42b5e9558def271,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"bb97b7c475a5d83a821ea720d61af589","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:54:20.299Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-041c6ce431c75be173757456: start","job_status":"start","queue_duration_s":98.51315,"scheduling_latency_s":98.513207}1193gitlab # [ 204.098976] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:55:59.035Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"041c6ce431c75be173757456","created_at":"2026-10-02T16:54:20.286Z","trace_propagation_headers":{"sentry-trace":"815d05d9ebf949aaa42b5e9558def271-416cca0e315142ec","baggage":"sentry-trace_id=815d05d9ebf949aaa42b5e9558def271,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"cell","correlation_id":"bb97b7c475a5d83a821ea720d61af589","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:54:20.299Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-041c6ce431c75be173757456: done: 0.222906 sec","job_status":"done","queue_duration_s":98.51315,"scheduling_latency_s":98.513207,"redis_calls":29,"redis_duration_s":0.006307999999999999,"redis_read_bytes":282,"redis_write_bytes":3632,"redis_feature_flag_calls":18,"redis_feature_flag_duration_s":0.003737,"redis_feature_flag_read_bytes":272,"redis_feature_flag_write_bytes":2385,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.00155,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":688,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":559,"db_count":15,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.108781,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.12042696,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.222906,"completed_at":"2026-10-02T16:55:59.035Z","load_balancing_strategy":"primary","db_duration_s":0.062723,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1194gitlab # [ 208.458713] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.394Z","pid":1860,"message":"* Listening on unix:///var/gitlab/state//tmp/sockets/gitlab.socket"}1195gitlab # [ 208.458927] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.394Z","pid":1860,"message":"! WARNING: Detected 2 Thread(s) started in app boot:"}1196gitlab # [ 208.459025] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.394Z","pid":1860,"message":"! #\u003cThread:0x0000ec4e23546240 /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:72 sleep\u003e - /nix/store/zj7sa5zsg6vhnyq28b8v0x1jajz1gpvc-ruby3.3-rack-timeout-0.7.0/lib/ruby/gems/3.3.0/gems/rack-timeout-0.7.0/lib/rack/timeout/support/scheduler.rb:90:in `sleep'"}1197gitlab # [ 208.459096] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.394Z","pid":1860,"message":"! #\u003cThread:0x0000ec4e237414c8 /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:18 sleep\u003e - /nix/store/lnly88svq5ykjs8ynaw5cm3g88z1lvsm-ruby3.3-sentry-ruby-5.23.0/lib/ruby/gems/3.3.0/gems/sentry-ruby-5.23.0/lib/sentry/threaded_periodic_worker.rb:20:in `sleep'"}1198gitlab # [ 208.459161] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.395Z","pid":1860,"message":"Use Ctrl-C to stop"}1199gitlab # [ 208.459214] bundle[1860]: {"timestamp":"2026-10-02T16:56:03.395Z","pid":1860,"message":"Sending status to systemd every 1.0 sec"}1200gitlab # [ 212.704220] bundle[1860]: {"timestamp":"2026-10-02T16:56:07.639Z","pid":1860,"message":"- Worker 0 (PID: 2810) booted in 4.2s, phase: 0"}1201gitlab # [ 212.706904] bundle[1860]: {"timestamp":"2026-10-02T16:56:07.643Z","pid":1860,"message":"- Worker 1 (PID: 2818) booted in 4.17s, phase: 0"}1202gitlab # [ 212.711141] systemd[1]: Started gitlab.service.1203gitlab # [ 212.711994] systemd[1]: Reached target Common target for all GitLab services..1204gitlab # [ 212.714725] systemd[1]: Starting setup-gitlab.service...1205gitlab # [ 212.742496] setup-gitlab-start[2878]: + timeout 900 bash -c 'until curl -fs http://gitlab/users/sign_in > /dev/null; do sleep 5; done'1206gitlab # [ 213.634329] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:56:08 +0000] "GET /users/sign_in HTTP/1.1" 200 10655 "" "curl/8.21.0" 8641207gitlab # [ 213.637913] setup-gitlab-start[2878]: + TOKEN=nixbot-test-token-of-20-chars1208gitlab # [ 213.639802] setup-gitlab-start[2878]: + runuser -u gitlab -- /run/current-system/sw/bin/gitlab-rails runner $'\n ApplicationSetting.current.update!(allow_local_requests_from_web_hooks_and_services: true)\n Rails.cache.clear\n token = User.find_by_username(\'root\').personal_access_tokens.create!(\n name: \'nixbot\', scopes: [:api], expires_at: 7.days.from_now)\n token.set_token(\'nixbot-test-token-of-20-chars\')\n token.save!\n'1209gitlab # [ 213.649408] runuser[2893]: pam_unix(runuser:session): session opened for user gitlab(uid=165) by (uid=0)1210gitlab # [ 214.203194] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1211gitlab # [ 214.206921] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1212gitlab # [ 214.210394] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1213gitlab # [ 215.071216] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=rbs version=3.4.0 platform=ruby> because it is missing extensions1214gitlab # [ 215.071600] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=racc version=1.7.3 platform=ruby> because it is missing extensions1215gitlab # [ 215.079197] setup-gitlab-start[2894]: Source locally installed gems is ignoring #<Bundler::StubSpecification name=debug version=1.9.2 platform=ruby> because it is missing extensions1216gitlab # [ 216.956525] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:11.891Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"adc50c4b4aa296bbf7375412","created_at":"2026-10-02T16:56:11.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"2e0e9f7a50f554cff690abed93c45932","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.889Z","job_size_bytes":3,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-adc50c4b4aa296bbf7375412: start","job_status":"start","queue_duration_s":0.002519,"scheduling_latency_s":0.002567}1217gitlab # [ 217.075588] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.011Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":["1"],"class":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker","jid":"adc50c4b4aa296bbf7375412","created_at":"2026-10-02T16:56:11.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.feature_category":"fleet_visibility","correlation_id":"2e0e9f7a50f554cff690abed93c45932","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f3066c5baf1b66987ee2154bc897f8592e7a97485c950408f900b17a65777f1b","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.889Z","job_size_bytes":3,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ClickHouse::FinishedPipelinesSyncCronWorker JID-adc50c4b4aa296bbf7375412: done: 0.119773 sec","job_status":"done","queue_duration_s":0.002519,"scheduling_latency_s":0.002567,"redis_calls":10,"redis_duration_s":0.009132,"redis_read_bytes":9,"redis_write_bytes":1203,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000542,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007497,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024686,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00548098,"duration_s":0.119773,"completed_at":"2026-10-02T16:56:12.011Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.015838,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1218gitlab # [ 217.139521] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.018Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5f17d2340932ceb71d2d62db","created_at":"2026-10-02T16:56:11.902Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f58c154540f93a7afa235498cfb198d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.924Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5f17d2340932ceb71d2d62db: start","job_status":"start","queue_duration_s":0.093536,"scheduling_latency_s":0.093572}1219gitlab # [ 217.210424] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.145Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"5f17d2340932ceb71d2d62db","created_at":"2026-10-02T16:56:11.902Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"7f58c154540f93a7afa235498cfb198d","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.924Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-5f17d2340932ceb71d2d62db: done: 0.126921 sec","job_status":"done","queue_duration_s":0.093536,"scheduling_latency_s":0.093572,"redis_calls":12,"redis_duration_s":0.010501,"redis_read_bytes":11,"redis_write_bytes":1489,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007002,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003047,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02887,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0008467,"exclusive_lock_hold_duration_s":0.032547320000000005,"duration_s":0.126921,"completed_at":"2026-10-02T16:56:12.144Z","load_balancing_strategy":"primary","db_duration_s":0.022884,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1220gitlab # [ 217.262904] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.150Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9e5f46f30239a0c55b732067","created_at":"2026-10-02T16:56:11.960Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2780b390d5c26148fb6aba652f047d6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.968Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9e5f46f30239a0c55b732067: start","job_status":"start","queue_duration_s":0.182413,"scheduling_latency_s":0.182445}1221gitlab # [ 217.300346] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.236Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"9e5f46f30239a0c55b732067","created_at":"2026-10-02T16:56:11.960Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b2780b390d5c26148fb6aba652f047d6","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.968Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-9e5f46f30239a0c55b732067: done: 0.085774 sec","job_status":"done","queue_duration_s":0.182413,"scheduling_latency_s":0.182445,"redis_calls":12,"redis_duration_s":0.025537999999999998,"redis_read_bytes":11,"redis_write_bytes":1511,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001778,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.018871,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004889,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00109454,"exclusive_lock_hold_duration_s":0.01194222,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.085774,"completed_at":"2026-10-02T16:56:12.236Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01379,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1222gitlab # [ 217.358286] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.243Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1a387dd892292a46d55de26f","created_at":"2026-10-02T16:56:11.973Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c25e7c66309cb21924f1a49a7eb76a97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.980Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1a387dd892292a46d55de26f: start","job_status":"start","queue_duration_s":0.26332,"scheduling_latency_s":0.263357}1223gitlab # [ 217.396427] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.332Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"1a387dd892292a46d55de26f","created_at":"2026-10-02T16:56:11.973Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c25e7c66309cb21924f1a49a7eb76a97","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:11.980Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-1a387dd892292a46d55de26f: done: 0.088581 sec","job_status":"done","queue_duration_s":0.26332,"scheduling_latency_s":0.263357,"redis_calls":12,"redis_duration_s":0.020087999999999998,"redis_read_bytes":11,"redis_write_bytes":1566,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00643,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012903,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":670,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000755,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00382412,"duration_s":0.088581,"completed_at":"2026-10-02T16:56:12.332Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005499,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1224gitlab # [ 217.448064] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.340Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ed26a438780bdc79ddfd2c4f","created_at":"2026-10-02T16:56:11.997Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d96631e84664e333b52d2236910b048","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.014Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ed26a438780bdc79ddfd2c4f: start","job_status":"start","queue_duration_s":0.32672,"scheduling_latency_s":0.326761}1225gitlab # [ 217.462929] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.367Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1226gitlab # [ 217.466431] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.390Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"ed26a438780bdc79ddfd2c4f","created_at":"2026-10-02T16:56:11.997Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5d96631e84664e333b52d2236910b048","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.014Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-ed26a438780bdc79ddfd2c4f: done: 0.049736 sec","job_status":"done","queue_duration_s":0.32672,"scheduling_latency_s":0.326761,"redis_calls":10,"redis_duration_s":0.010799,"redis_read_bytes":9,"redis_write_bytes":1207,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.003266,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004147,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00232812,"duration_s":0.049736,"completed_at":"2026-10-02T16:56:12.390Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00059,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1227gitlab # [ 217.510097] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.394Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e335fc732932ff96a6465903","created_at":"2026-10-02T16:56:12.046Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04ddd29b0021e08a12c3b6f00e8c65b1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.079Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e335fc732932ff96a6465903: start","job_status":"start","queue_duration_s":0.314864,"scheduling_latency_s":0.314895}1228gitlab # [ 217.747669] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::UpdateLockedUnknownArtifactsWorker","jid":"e335fc732932ff96a6465903","created_at":"2026-10-02T16:56:12.046Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"04ddd29b0021e08a12c3b6f00e8c65b1","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:912858db94fb8c9da5e144f8394730b16dbb4d7f396606379d4f42078eef6e20","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.079Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UpdateLockedUnknownArtifactsWorker JID-e335fc732932ff96a6465903: done: 0.289652 sec","job_status":"done","queue_duration_s":0.314864,"scheduling_latency_s":0.314895,"redis_calls":12,"redis_duration_s":0.01783,"redis_read_bytes":11,"redis_write_bytes":1447,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002256,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":319,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007411,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":586,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.008163,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":542,"db_count":101,"db_write_count":0,"db_cached_count":99,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":101,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":99,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":100,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":99,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.014,"db_ci_duration_s":0.015,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.193495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00281936,"exclusive_lock_hold_duration_s":0.20457028,"extra.ci_update_locked_unknown_artifacts_worker.removed_count":0,"extra.ci_update_locked_unknown_artifacts_worker.locked_count":0,"duration_s":0.289652,"completed_at":"2026-10-02T16:56:12.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.045827,"urgency":"throttled","target_duration_s":300,"target_scheduling_latency_s":null}1229gitlab # [ 217.796443] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.706Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"af5289556806ea3a906e594e","created_at":"2026-10-02T16:56:12.096Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62ebaf14e15b92164639eef07e4c132","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.107Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-af5289556806ea3a906e594e: start","job_status":"start","queue_duration_s":0.598442,"scheduling_latency_s":0.59848}1230gitlab # [ 217.841587] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ServiceDesk::CustomEmailVerificationCleanupWorker","jid":"af5289556806ea3a906e594e","created_at":"2026-10-02T16:56:12.096Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d62ebaf14e15b92164639eef07e4c132","meta.root_caller_id":"Cronjob","meta.feature_category":"service_desk","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:521f52c3ade07cc9136cad2d9aca0ecb702f305a24b32b7c6ca1558624b4df26","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.107Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ServiceDesk::CustomEmailVerificationCleanupWorker JID-af5289556806ea3a906e594e: done: 0.07174 sec","job_status":"done","queue_duration_s":0.598442,"scheduling_latency_s":0.59848,"redis_calls":10,"redis_duration_s":0.016608,"redis_read_bytes":9,"redis_write_bytes":1225,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002973,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.007399,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.006236,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":250,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028446,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00135192,"duration_s":0.07174,"completed_at":"2026-10-02T16:56:12.777Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.011255,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1231gitlab # [ 217.885470] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.784Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d1da9b3a1748fc7b90e867b6","created_at":"2026-10-02T16:56:12.141Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26915e92a8c516453332ba146074b1a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.162Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d1da9b3a1748fc7b90e867b6: start","job_status":"start","queue_duration_s":0.621919,"scheduling_latency_s":0.621949}1232gitlab # [ 217.918250] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.854Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"d1da9b3a1748fc7b90e867b6","created_at":"2026-10-02T16:56:12.141Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"26915e92a8c516453332ba146074b1a2","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.162Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-d1da9b3a1748fc7b90e867b6: done: 0.06973 sec","job_status":"done","queue_duration_s":0.621919,"scheduling_latency_s":0.621949,"redis_calls":10,"redis_duration_s":0.015613,"redis_read_bytes":9,"redis_write_bytes":1215,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001414,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.012759,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":626,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00144,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0005988,"duration_s":0.06973,"completed_at":"2026-10-02T16:56:12.854Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003088,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1233gitlab # [ 217.965629] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:12.863Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fcf4f1182cb95578771e90dd","created_at":"2026-10-02T16:56:12.178Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"514367803d272c6f30de08adb62866f0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.198Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fcf4f1182cb95578771e90dd: start","job_status":"start","queue_duration_s":0.664678,"scheduling_latency_s":0.66471}1234gitlab # [ 218.317679] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.253Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"fcf4f1182cb95578771e90dd","created_at":"2026-10-02T16:56:12.178Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"514367803d272c6f30de08adb62866f0","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.198Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-fcf4f1182cb95578771e90dd: done: 0.390049 sec","job_status":"done","queue_duration_s":0.664678,"scheduling_latency_s":0.66471,"redis_calls":565,"redis_duration_s":0.07654999999999999,"redis_read_bytes":564,"redis_write_bytes":49139,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000386,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001457,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":546,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.074707,"redis_shared_state_read_bytes":558,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.314419,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00032048,"duration_s":0.390049,"completed_at":"2026-10-02T16:56:13.253Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1235gitlab # [ 218.375601] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.255Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0e45df401ffeff5f59f4ee6","created_at":"2026-10-02T16:56:12.228Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.241Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0e45df401ffeff5f59f4ee6: start","job_status":"start","queue_duration_s":1.013927,"scheduling_latency_s":1.013951}1236gitlab # [ 218.441309] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.377Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"a0e45df401ffeff5f59f4ee6","created_at":"2026-10-02T16:56:12.228Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.241Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-a0e45df401ffeff5f59f4ee6: done: 0.12197 sec","job_status":"done","queue_duration_s":1.013927,"scheduling_latency_s":1.013951,"redis_calls":71,"redis_duration_s":0.012688,"redis_read_bytes":120,"redis_write_bytes":42674,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001872,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.001026,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":56,"redis_queues_metadata_duration_s":0.009226,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3418,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000564,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":300,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.106761,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029262,"duration_s":0.12197,"completed_at":"2026-10-02T16:56:13.377Z","load_balancing_strategy":"primary","db_duration_s":0.002246,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1237gitlab # [ 218.503434] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"164b5048f05b66d58f35183d","created_at":"2026-10-02T16:56:12.259Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ac32e0fe2ed43eea997f419232c951","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.282Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-164b5048f05b66d58f35183d: start","job_status":"start","queue_duration_s":1.096903,"scheduling_latency_s":1.096924}1238gitlab # [ 218.722179] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.658Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"164b5048f05b66d58f35183d","created_at":"2026-10-02T16:56:12.259Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a2ac32e0fe2ed43eea997f419232c951","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.282Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-164b5048f05b66d58f35183d: done: 0.279232 sec","job_status":"done","queue_duration_s":1.096903,"scheduling_latency_s":1.096924,"redis_calls":14,"redis_duration_s":0.002451,"redis_read_bytes":13,"redis_write_bytes":1645,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000601,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":580,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000998,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000852,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.034,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":131,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":84,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.034,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.234991,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.0005711,"exclusive_lock_hold_duration_s":0.25304962,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"main","turbo_mode":false},"duration_s":0.279232,"completed_at":"2026-10-02T16:56:13.658Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.050052,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1239gitlab # [ 218.790851] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"99e64fa6951403698090f3c7","created_at":"2026-10-02T16:56:12.301Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90f0f83d41d3b616d5beb838005d8a91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.318Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-99e64fa6951403698090f3c7: start","job_status":"start","queue_duration_s":1.342572,"scheduling_latency_s":1.342597}1240gitlab # [ 218.810729] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"99e64fa6951403698090f3c7","created_at":"2026-10-02T16:56:12.301Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"90f0f83d41d3b616d5beb838005d8a91","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.318Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-99e64fa6951403698090f3c7: done: 0.054741 sec","job_status":"done","queue_duration_s":1.342572,"scheduling_latency_s":1.342597,"redis_calls":12,"redis_duration_s":0.0028640000000000002,"redis_read_bytes":11,"redis_write_bytes":1622,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000391,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001309,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":658,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.017,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00084372,"exclusive_lock_hold_duration_s":0.016719139999999997,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.054741,"completed_at":"2026-10-02T16:56:13.715Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.019594,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1241gitlab # [ 218.862289] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.717Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6b1cb866b3ed419bfa7dc31c","created_at":"2026-10-02T16:56:12.330Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fb12feebfc415549e248f54cc9dbb13","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.343Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6b1cb866b3ed419bfa7dc31c: start","job_status":"start","queue_duration_s":1.374171,"scheduling_latency_s":1.374202}1242gitlab # [ 218.877123] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.752Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"6b1cb866b3ed419bfa7dc31c","created_at":"2026-10-02T16:56:12.330Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8fb12feebfc415549e248f54cc9dbb13","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.343Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-6b1cb866b3ed419bfa7dc31c: done: 0.035149 sec","job_status":"done","queue_duration_s":1.374171,"scheduling_latency_s":1.374202,"redis_calls":10,"redis_duration_s":0.002664,"redis_read_bytes":9,"redis_write_bytes":1061,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000655,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00151,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":550,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003454,"duration_s":0.035149,"completed_at":"2026-10-02T16:56:13.752Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003159,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1243gitlab # [ 218.920674] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.754Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8db50804756e99be41cc3e80","created_at":"2026-10-02T16:56:12.368Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82d4150923c489a82150c53dcb5a6bc1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.387Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8db50804756e99be41cc3e80: start","job_status":"start","queue_duration_s":1.366872,"scheduling_latency_s":1.366895}1244gitlab # [ 218.935819] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.768Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1245gitlab # [ 218.939793] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.776Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"8db50804756e99be41cc3e80","created_at":"2026-10-02T16:56:12.368Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82d4150923c489a82150c53dcb5a6bc1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.387Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-8db50804756e99be41cc3e80: done: 0.021937 sec","job_status":"done","queue_duration_s":1.366872,"scheduling_latency_s":1.366895,"redis_calls":10,"redis_duration_s":0.00182,"redis_read_bytes":9,"redis_write_bytes":1199,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00038,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":618,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000399,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012878,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028984,"duration_s":0.021937,"completed_at":"2026-10-02T16:56:13.776Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000456,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1246gitlab # [ 218.984841] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.777Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7db8239734fdb69a83d26f58","created_at":"2026-10-02T16:56:12.399Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88131621bfe1c3d1ca366fdba5158410","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.417Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7db8239734fdb69a83d26f58: start","job_status":"start","queue_duration_s":1.360693,"scheduling_latency_s":1.360712}1247gitlab # [ 219.001487] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.790Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1248gitlab # [ 219.005145] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.798Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"7db8239734fdb69a83d26f58","created_at":"2026-10-02T16:56:12.399Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"88131621bfe1c3d1ca366fdba5158410","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.417Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-7db8239734fdb69a83d26f58: done: 0.020645 sec","job_status":"done","queue_duration_s":1.360693,"scheduling_latency_s":1.360712,"redis_calls":10,"redis_duration_s":0.001981,"redis_read_bytes":9,"redis_write_bytes":1291,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000618,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011929,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026938,"duration_s":0.020645,"completed_at":"2026-10-02T16:56:13.798Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000366,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1249gitlab # [ 219.048987] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.799Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7d3ac4ff1c43bf130394c046","created_at":"2026-10-02T16:56:12.435Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.453Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7d3ac4ff1c43bf130394c046: start","job_status":"start","queue_duration_s":1.346405,"scheduling_latency_s":1.346424}1250gitlab # [ 219.063010] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.975Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"7d3ac4ff1c43bf130394c046","created_at":"2026-10-02T16:56:12.435Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.453Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-7d3ac4ff1c43bf130394c046: done: 0.175886 sec","job_status":"done","queue_duration_s":1.346405,"scheduling_latency_s":1.346424,"redis_calls":19,"redis_duration_s":0.003468,"redis_read_bytes":22,"redis_write_bytes":4789,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000887,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000285,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.001698,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":814,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000598,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027448,"duration_s":0.175886,"completed_at":"2026-10-02T16:56:13.975Z","load_balancing_strategy":"primary","db_duration_s":0.002704,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1251gitlab # [ 219.107405] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"857551afbe8f261469584701","created_at":"2026-10-02T16:56:12.464Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7c0e9026cbbbaf8548acf641784de0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:56:12.475Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-857551afbe8f261469584701: start","job_status":"start","queue_duration_s":1.501605,"scheduling_latency_s":1.501627}1252gitlab # [ 219.121492] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.989Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1253gitlab # [ 219.125317] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.997Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"857551afbe8f261469584701","created_at":"2026-10-02T16:56:12.464Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4f7c0e9026cbbbaf8548acf641784de0","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:56:12.475Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-857551afbe8f261469584701: done: 0.020332 sec","job_status":"done","queue_duration_s":1.501605,"scheduling_latency_s":1.501627,"redis_calls":10,"redis_duration_s":0.002066,"redis_read_bytes":9,"redis_write_bytes":1253,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000652,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":646,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000371,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011471,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0002792,"duration_s":0.020332,"completed_at":"2026-10-02T16:56:13.997Z","load_balancing_strategy":"primary","db_duration_s":0.000584,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1254gitlab # [ 219.167769] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:13.999Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"75346db17fe4640e0d5bbc80","created_at":"2026-10-02T16:56:12.495Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de281d69aff30bf0e14d87558c4fa1f0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.656Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-75346db17fe4640e0d5bbc80: start","job_status":"start","queue_duration_s":1.342476,"scheduling_latency_s":1.342495}1255gitlab # [ 219.182378] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.036Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"75346db17fe4640e0d5bbc80","created_at":"2026-10-02T16:56:12.495Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"de281d69aff30bf0e14d87558c4fa1f0","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.656Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-75346db17fe4640e0d5bbc80: done: 0.037028 sec","job_status":"done","queue_duration_s":1.342476,"scheduling_latency_s":1.342495,"redis_calls":10,"redis_duration_s":0.001898,"redis_read_bytes":9,"redis_write_bytes":1289,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000404,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001118,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":666,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015383,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027868,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.037028,"completed_at":"2026-10-02T16:56:14.035Z","load_balancing_strategy":"primary","db_duration_s":0.014203,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1256gitlab # [ 219.227971] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.037Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ee9440c029e75373089f9744","created_at":"2026-10-02T16:56:12.680Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cf9c45f677f2af6036368faa1712973","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.689Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ee9440c029e75373089f9744: start","job_status":"start","queue_duration_s":1.347676,"scheduling_latency_s":1.347694}1257gitlab # [ 219.243376] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.062Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Users::MigrateRecordsToGhostUserInBatchesWorker","jid":"ee9440c029e75373089f9744","created_at":"2026-10-02T16:56:12.680Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"0cf9c45f677f2af6036368faa1712973","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4bdb3193c92ce2ad4e73652ad55816e507f3dd7f07576150fc7c08572353b9e8","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.689Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Users::MigrateRecordsToGhostUserInBatchesWorker JID-ee9440c029e75373089f9744: done: 0.025181 sec","job_status":"done","queue_duration_s":1.347676,"scheduling_latency_s":1.347694,"redis_calls":12,"redis_duration_s":0.002291,"redis_read_bytes":11,"redis_write_bytes":1562,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000382,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001043,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":634,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000866,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":591,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014358,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052966,"exclusive_lock_hold_duration_s":0.00534108,"duration_s":0.025181,"completed_at":"2026-10-02T16:56:14.062Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00265,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1258gitlab # [ 219.287489] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6f4517539591535149c54f09","created_at":"2026-10-02T16:56:12.702Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"814b7ec0e7cf535fee20c541ce5840ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:56:12.712Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6f4517539591535149c54f09: start","job_status":"start","queue_duration_s":1.350936,"scheduling_latency_s":1.350954}1259gitlab # [ 219.301821] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.077Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1260gitlab # [ 219.305339] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.085Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"6f4517539591535149c54f09","created_at":"2026-10-02T16:56:12.702Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"814b7ec0e7cf535fee20c541ce5840ee","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:56:12.712Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-6f4517539591535149c54f09: done: 0.021752 sec","job_status":"done","queue_duration_s":1.350936,"scheduling_latency_s":1.350954,"redis_calls":10,"redis_duration_s":0.001931,"redis_read_bytes":9,"redis_write_bytes":1261,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00108,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":650,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000439,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012719,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00027016,"duration_s":0.021752,"completed_at":"2026-10-02T16:56:14.085Z","load_balancing_strategy":"primary","db_duration_s":0.000495,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1261gitlab # [ 219.347925] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.086Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"66b928cc623bd41c3fca1f17","created_at":"2026-10-02T16:56:12.730Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60c02431bdb2e0f2e95384bc9a880fe4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.749Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-66b928cc623bd41c3fca1f17: start","job_status":"start","queue_duration_s":1.336955,"scheduling_latency_s":1.336972}1262gitlab # [ 219.363287] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.133Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"66b928cc623bd41c3fca1f17","created_at":"2026-10-02T16:56:12.730Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"60c02431bdb2e0f2e95384bc9a880fe4","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.749Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-66b928cc623bd41c3fca1f17: done: 0.047089 sec","job_status":"done","queue_duration_s":1.336955,"scheduling_latency_s":1.336972,"redis_calls":14,"redis_duration_s":0.002659,"redis_read_bytes":13,"redis_write_bytes":1883,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000697,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":618,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001033,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":654,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":611,"db_count":8,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00055416,"exclusive_lock_hold_duration_s":0.024996420000000002,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.047089,"completed_at":"2026-10-02T16:56:14.133Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014613,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1263gitlab # [ 219.411224] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.135Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"74e0332b183c76e1707f4121","created_at":"2026-10-02T16:56:12.762Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f01e6d4dcf484698c31eeece441f087","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.780Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-74e0332b183c76e1707f4121: start","job_status":"start","queue_duration_s":1.354543,"scheduling_latency_s":1.354561}1264gitlab # [ 219.426512] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.167Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"74e0332b183c76e1707f4121","created_at":"2026-10-02T16:56:12.762Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"2f01e6d4dcf484698c31eeece441f087","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.780Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-74e0332b183c76e1707f4121: done: 0.032192 sec","job_status":"done","queue_duration_s":1.354543,"scheduling_latency_s":1.354561,"redis_calls":13,"redis_duration_s":0.002331,"redis_read_bytes":12,"redis_write_bytes":1668,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001186,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":728,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01048608,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.032192,"completed_at":"2026-10-02T16:56:14.167Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002891,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1265gitlab # [ 219.472360] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.168Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"60d33707729a97d8a3425286","created_at":"2026-10-02T16:56:12.796Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"287992df6d3403440c461855c6ea5dae","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.803Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-60d33707729a97d8a3425286: start","job_status":"start","queue_duration_s":1.365211,"scheduling_latency_s":1.365231}1266gitlab # [ 219.486029] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.191Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"GitlabPerformanceBarStatsWorker","jid":"60d33707729a97d8a3425286","created_at":"2026-10-02T16:56:12.796Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"287992df6d3403440c461855c6ea5dae","meta.root_caller_id":"Cronjob","meta.feature_category":"observability","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:393c75c82e0c4ccb08a7f1ba676a44e17ea4991969dda9da33b08a8d5bf6f24b","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.803Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"GitlabPerformanceBarStatsWorker JID-60d33707729a97d8a3425286: done: 0.022988 sec","job_status":"done","queue_duration_s":1.365211,"scheduling_latency_s":1.365231,"redis_calls":12,"redis_duration_s":0.001932,"redis_read_bytes":10,"redis_write_bytes":1174,"redis_cache_calls":2,"redis_cache_duration_s":0.000423,"redis_cache_read_bytes":1,"redis_cache_write_bytes":93,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000353,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":562,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":214,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013266,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00024404,"duration_s":0.022988,"completed_at":"2026-10-02T16:56:14.191Z","load_balancing_strategy":"primary","db_duration_s":0.002743,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1267gitlab # [ 219.529952] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.193Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"729b4be16b56d8c1b3084456","created_at":"2026-10-02T16:56:12.826Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64b7c4cf951eb19b61e412b0f8d13fad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.851Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-729b4be16b56d8c1b3084456: start","job_status":"start","queue_duration_s":1.342022,"scheduling_latency_s":1.342041}1268gitlab # [ 219.544701] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.225Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"729b4be16b56d8c1b3084456","created_at":"2026-10-02T16:56:12.826Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"64b7c4cf951eb19b61e412b0f8d13fad","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.851Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-729b4be16b56d8c1b3084456: done: 0.032718 sec","job_status":"done","queue_duration_s":1.342022,"scheduling_latency_s":1.342041,"redis_calls":16,"redis_duration_s":0.002415,"redis_read_bytes":15,"redis_write_bytes":1942,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.001247,"redis_feature_flag_read_bytes":8,"redis_feature_flag_write_bytes":1148,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000831,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":574,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":220,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00028298,"duration_s":0.032718,"completed_at":"2026-10-02T16:56:14.225Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004226,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1269gitlab # [ 219.587971] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.227Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3d37f007c90c1a8a75931903","created_at":"2026-10-02T16:56:12.860Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c70d0f46e4d6da808eee133c4457258","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:56:12.872Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3d37f007c90c1a8a75931903: start","job_status":"start","queue_duration_s":1.354995,"scheduling_latency_s":1.355013}1270gitlab # [ 219.601843] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.262Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"3d37f007c90c1a8a75931903","created_at":"2026-10-02T16:56:12.860Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9c70d0f46e4d6da808eee133c4457258","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:56:12.872Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-3d37f007c90c1a8a75931903: done: 0.035815 sec","job_status":"done","queue_duration_s":1.354995,"scheduling_latency_s":1.355013,"redis_calls":10,"redis_duration_s":0.0016660000000000002,"redis_read_bytes":9,"redis_write_bytes":1157,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000321,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000994,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000351,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022616,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0003281,"duration_s":0.035815,"completed_at":"2026-10-02T16:56:14.262Z","load_balancing_strategy":"primary","db_duration_s":0.003898,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1271gitlab # [ 219.645798] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.264Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"84107c3c3e2f1d217bf40c6a","created_at":"2026-10-02T16:56:12.891Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a018e497cad08479005cef2f0a52a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.909Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-84107c3c3e2f1d217bf40c6a: start","job_status":"start","queue_duration_s":1.354474,"scheduling_latency_s":1.354494}1272gitlab # [ 219.661068] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.275Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1273gitlab # [ 219.665052] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.283Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"84107c3c3e2f1d217bf40c6a","created_at":"2026-10-02T16:56:12.891Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4a018e497cad08479005cef2f0a52a6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:56:12.909Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-84107c3c3e2f1d217bf40c6a: done: 0.019347 sec","job_status":"done","queue_duration_s":1.354474,"scheduling_latency_s":1.354494,"redis_calls":10,"redis_duration_s":0.0014939999999999999,"redis_read_bytes":9,"redis_write_bytes":1283,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000332,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000823,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":662,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011317,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025662,"duration_s":0.019347,"completed_at":"2026-10-02T16:56:14.283Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000503,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1274gitlab # [ 219.709031] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.284Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5396acdc696ab9ebb481dde6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5396acdc696ab9ebb481dde6: start","job_status":"start","queue_duration_s":0.91775,"scheduling_latency_s":0.917791}1275gitlab # [ 219.722292] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.309Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5396acdc696ab9ebb481dde6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5396acdc696ab9ebb481dde6: done: 0.025225 sec","job_status":"done","queue_duration_s":0.91775,"scheduling_latency_s":0.917791,"redis_calls":16,"redis_duration_s":0.0027530000000000002,"redis_read_bytes":14,"redis_write_bytes":1544,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000322,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000702,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":439,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001729,"redis_shared_state_read_bytes":9,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015081,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00026992,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025225,"completed_at":"2026-10-02T16:56:14.309Z","load_balancing_strategy":"primary","db_duration_s":0.00039,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1276gitlab # [ 219.764791] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.311Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c22326d7e78a32db91d365df","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c22326d7e78a32db91d365df: start","job_status":"start","queue_duration_s":0.944179,"scheduling_latency_s":0.944197}1277gitlab # [ 219.777665] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.326Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c22326d7e78a32db91d365df","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c22326d7e78a32db91d365df: done: 0.014889 sec","job_status":"done","queue_duration_s":0.944179,"scheduling_latency_s":0.944197,"redis_calls":13,"redis_duration_s":0.001534,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000565,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01025,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014889,"completed_at":"2026-10-02T16:56:14.326Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1278gitlab # [ 219.819081] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.327Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61b404c333e394364f732183","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61b404c333e394364f732183: start","job_status":"start","queue_duration_s":0.960315,"scheduling_latency_s":0.960333}1279gitlab # [ 219.832278] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.343Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"61b404c333e394364f732183","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-61b404c333e394364f732183: done: 0.015895 sec","job_status":"done","queue_duration_s":0.960315,"scheduling_latency_s":0.960333,"redis_calls":13,"redis_duration_s":0.0016060000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010318,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015895,"completed_at":"2026-10-02T16:56:14.343Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1280gitlab # [ 219.872964] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.345Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0201b8c90d81eeae98f6a169","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0201b8c90d81eeae98f6a169: start","job_status":"start","queue_duration_s":0.977411,"scheduling_latency_s":0.978256}1281gitlab # [ 219.885867] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.361Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0201b8c90d81eeae98f6a169","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0201b8c90d81eeae98f6a169: done: 0.016917 sec","job_status":"done","queue_duration_s":0.977411,"scheduling_latency_s":0.978256,"redis_calls":13,"redis_duration_s":0.0015370000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010078,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016917,"completed_at":"2026-10-02T16:56:14.361Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1282gitlab # [ 219.926575] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.362Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"432102c76efae03534ad16b3","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-432102c76efae03534ad16b3: start","job_status":"start","queue_duration_s":0.995553,"scheduling_latency_s":0.995571}1283gitlab # [ 219.939578] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.379Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"432102c76efae03534ad16b3","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-432102c76efae03534ad16b3: done: 0.016485 sec","job_status":"done","queue_duration_s":0.995553,"scheduling_latency_s":0.995571,"redis_calls":13,"redis_duration_s":0.001583,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000598,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000985,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011849,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016485,"completed_at":"2026-10-02T16:56:14.378Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1284gitlab # [ 219.980779] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.380Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d0a635aec5bf12964f35e9","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d0a635aec5bf12964f35e9: start","job_status":"start","queue_duration_s":1.013214,"scheduling_latency_s":1.013232}1285gitlab # [ 219.993787] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.397Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"28d0a635aec5bf12964f35e9","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-28d0a635aec5bf12964f35e9: done: 0.017075 sec","job_status":"done","queue_duration_s":1.013214,"scheduling_latency_s":1.013232,"redis_calls":13,"redis_duration_s":0.0018769999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00061,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001267,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017075,"completed_at":"2026-10-02T16:56:14.397Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1286gitlab # [ 220.036344] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.398Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7db36909cbc21c9c7a4a9eb","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7db36909cbc21c9c7a4a9eb: start","job_status":"start","queue_duration_s":1.03144,"scheduling_latency_s":1.031457}1287gitlab # [ 220.048918] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.415Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d7db36909cbc21c9c7a4a9eb","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d7db36909cbc21c9c7a4a9eb: done: 0.017038 sec","job_status":"done","queue_duration_s":1.03144,"scheduling_latency_s":1.031457,"redis_calls":13,"redis_duration_s":0.0016120000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001031,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01073,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017038,"completed_at":"2026-10-02T16:56:14.415Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1288gitlab # [ 220.089789] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.418Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a81c1eeb6d89a226845dae11","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a81c1eeb6d89a226845dae11: start","job_status":"start","queue_duration_s":1.050957,"scheduling_latency_s":1.050975}1289gitlab # [ 220.102639] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.435Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a81c1eeb6d89a226845dae11","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a81c1eeb6d89a226845dae11: done: 0.017392 sec","job_status":"done","queue_duration_s":1.050957,"scheduling_latency_s":1.050975,"redis_calls":13,"redis_duration_s":0.0026030000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000622,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017392,"completed_at":"2026-10-02T16:56:14.435Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1290gitlab # [ 220.144931] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.436Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04d21ac81e09addcf2b9ffff","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04d21ac81e09addcf2b9ffff: start","job_status":"start","queue_duration_s":1.06952,"scheduling_latency_s":1.069537}1291gitlab # [ 220.157734] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.451Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"04d21ac81e09addcf2b9ffff","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-04d21ac81e09addcf2b9ffff: done: 0.015299 sec","job_status":"done","queue_duration_s":1.06952,"scheduling_latency_s":1.069537,"redis_calls":13,"redis_duration_s":0.0021650000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001618,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010771,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015299,"completed_at":"2026-10-02T16:56:14.451Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1292gitlab # [ 220.198714] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.453Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f2b84f8957d2e2a7c0279f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f2b84f8957d2e2a7c0279f: start","job_status":"start","queue_duration_s":1.08603,"scheduling_latency_s":1.086049}1293gitlab # [ 220.211559] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.468Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0f2b84f8957d2e2a7c0279f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0f2b84f8957d2e2a7c0279f: done: 0.015472 sec","job_status":"done","queue_duration_s":1.08603,"scheduling_latency_s":1.086049,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015472,"completed_at":"2026-10-02T16:56:14.468Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1294gitlab # [ 220.253917] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33079d78be31d6e0dedad6df","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33079d78be31d6e0dedad6df: start","job_status":"start","queue_duration_s":1.102674,"scheduling_latency_s":1.102691}1295gitlab # [ 220.266793] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.484Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"33079d78be31d6e0dedad6df","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-33079d78be31d6e0dedad6df: done: 0.015259 sec","job_status":"done","queue_duration_s":1.102674,"scheduling_latency_s":1.102691,"redis_calls":13,"redis_duration_s":0.001542,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000958,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009919,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015259,"completed_at":"2026-10-02T16:56:14.484Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1296gitlab # [ 220.308008] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.486Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e256089d18965269ecc08284","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e256089d18965269ecc08284: start","job_status":"start","queue_duration_s":1.119655,"scheduling_latency_s":1.119673}1297gitlab # [ 220.320955] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.502Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e256089d18965269ecc08284","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e256089d18965269ecc08284: done: 0.015988 sec","job_status":"done","queue_duration_s":1.119655,"scheduling_latency_s":1.119673,"redis_calls":13,"redis_duration_s":0.001867,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000626,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001241,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015988,"completed_at":"2026-10-02T16:56:14.502Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1298gitlab # [ 220.361462] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"322875b0d60c0a038ff01f4c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-322875b0d60c0a038ff01f4c: start","job_status":"start","queue_duration_s":1.136725,"scheduling_latency_s":1.136742}1299gitlab # [ 220.374567] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.519Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"322875b0d60c0a038ff01f4c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-322875b0d60c0a038ff01f4c: done: 0.015518 sec","job_status":"done","queue_duration_s":1.136725,"scheduling_latency_s":1.136742,"redis_calls":13,"redis_duration_s":0.001592,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000611,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000981,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010851,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015518,"completed_at":"2026-10-02T16:56:14.519Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1300gitlab # [ 220.415237] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8112a56840c193d213d65486","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8112a56840c193d213d65486: start","job_status":"start","queue_duration_s":1.153349,"scheduling_latency_s":1.153367}1301gitlab # [ 220.429354] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.535Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8112a56840c193d213d65486","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8112a56840c193d213d65486: done: 0.014994 sec","job_status":"done","queue_duration_s":1.153349,"scheduling_latency_s":1.153367,"redis_calls":13,"redis_duration_s":0.001543,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000987,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01053,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014994,"completed_at":"2026-10-02T16:56:14.535Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1302gitlab # [ 220.470131] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.536Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc48b7b6d81d20db24d4918","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc48b7b6d81d20db24d4918: start","job_status":"start","queue_duration_s":1.169439,"scheduling_latency_s":1.169456}1303gitlab # [ 220.483176] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.551Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcc48b7b6d81d20db24d4918","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcc48b7b6d81d20db24d4918: done: 0.01523 sec","job_status":"done","queue_duration_s":1.169439,"scheduling_latency_s":1.169456,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001003,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01523,"completed_at":"2026-10-02T16:56:14.551Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1304gitlab # [ 220.524197] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.552Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d9d37c1d29a44705e414982","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d9d37c1d29a44705e414982: start","job_status":"start","queue_duration_s":1.185831,"scheduling_latency_s":1.185849}1305gitlab # [ 220.537094] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.567Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3d9d37c1d29a44705e414982","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3d9d37c1d29a44705e414982: done: 0.014832 sec","job_status":"done","queue_duration_s":1.185831,"scheduling_latency_s":1.185849,"redis_calls":13,"redis_duration_s":0.0015730000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000563,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00101,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010399,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014832,"completed_at":"2026-10-02T16:56:14.567Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1306gitlab # [ 220.577882] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.568Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67e820104bb39df60f418407","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67e820104bb39df60f418407: start","job_status":"start","queue_duration_s":1.201829,"scheduling_latency_s":1.201848}1307gitlab # [ 220.590743] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.583Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"67e820104bb39df60f418407","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-67e820104bb39df60f418407: done: 0.014908 sec","job_status":"done","queue_duration_s":1.201829,"scheduling_latency_s":1.201848,"redis_calls":13,"redis_duration_s":0.00152,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014908,"completed_at":"2026-10-02T16:56:14.583Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1308gitlab # [ 220.631558] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.585Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ebf29cc5e1d19340342ef5","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ebf29cc5e1d19340342ef5: start","job_status":"start","queue_duration_s":1.217983,"scheduling_latency_s":1.218}1309gitlab # [ 220.645965] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.600Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"43ebf29cc5e1d19340342ef5","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-43ebf29cc5e1d19340342ef5: done: 0.01536 sec","job_status":"done","queue_duration_s":1.217983,"scheduling_latency_s":1.218,"redis_calls":13,"redis_duration_s":0.0015789999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000557,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001022,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010411,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01536,"completed_at":"2026-10-02T16:56:14.600Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1310gitlab # [ 220.686952] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.601Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f39b40c23fbe09f778c46f15","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f39b40c23fbe09f778c46f15: start","job_status":"start","queue_duration_s":1.234806,"scheduling_latency_s":1.234824}1311gitlab # [ 220.699857] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.616Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f39b40c23fbe09f778c46f15","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f39b40c23fbe09f778c46f15: done: 0.014785 sec","job_status":"done","queue_duration_s":1.234806,"scheduling_latency_s":1.234824,"redis_calls":13,"redis_duration_s":0.001518,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010269,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014785,"completed_at":"2026-10-02T16:56:14.616Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1312gitlab # [ 220.741253] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.617Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4c5055aef74d61f1afaceaf","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4c5055aef74d61f1afaceaf: start","job_status":"start","queue_duration_s":1.250806,"scheduling_latency_s":1.250823}1313gitlab # [ 220.754028] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.632Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d4c5055aef74d61f1afaceaf","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d4c5055aef74d61f1afaceaf: done: 0.014915 sec","job_status":"done","queue_duration_s":1.250806,"scheduling_latency_s":1.250823,"redis_calls":13,"redis_duration_s":0.001526,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000978,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010382,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014915,"completed_at":"2026-10-02T16:56:14.632Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1314gitlab # [ 220.794947] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.633Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efcc006280c01ea321dbd8b7","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efcc006280c01ea321dbd8b7: start","job_status":"start","queue_duration_s":1.266881,"scheduling_latency_s":1.266899}1315gitlab # [ 220.808115] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.649Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"efcc006280c01ea321dbd8b7","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-efcc006280c01ea321dbd8b7: done: 0.015153 sec","job_status":"done","queue_duration_s":1.266881,"scheduling_latency_s":1.266899,"redis_calls":13,"redis_duration_s":0.0015799999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000993,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010637,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015153,"completed_at":"2026-10-02T16:56:14.648Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1316gitlab # [ 220.850370] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.650Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edadbb797bc26fb11e263d0b","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edadbb797bc26fb11e263d0b: start","job_status":"start","queue_duration_s":1.283184,"scheduling_latency_s":1.283202}1317gitlab # [ 220.863433] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.664Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"edadbb797bc26fb11e263d0b","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-edadbb797bc26fb11e263d0b: done: 0.014793 sec","job_status":"done","queue_duration_s":1.283184,"scheduling_latency_s":1.283202,"redis_calls":13,"redis_duration_s":0.001504,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014793,"completed_at":"2026-10-02T16:56:14.664Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1318gitlab # [ 220.904282] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.666Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed4163bd3d31e00d54571f06","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed4163bd3d31e00d54571f06: start","job_status":"start","queue_duration_s":1.299316,"scheduling_latency_s":1.299333}1319gitlab # [ 220.917214] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.681Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed4163bd3d31e00d54571f06","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed4163bd3d31e00d54571f06: done: 0.015083 sec","job_status":"done","queue_duration_s":1.299316,"scheduling_latency_s":1.299333,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000954,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010466,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015083,"completed_at":"2026-10-02T16:56:14.681Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1320gitlab # [ 220.957798] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.682Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2203c270f28849bbcb3aa6d4","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2203c270f28849bbcb3aa6d4: start","job_status":"start","queue_duration_s":1.315604,"scheduling_latency_s":1.315621}1321gitlab # [ 220.970533] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.697Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2203c270f28849bbcb3aa6d4","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2203c270f28849bbcb3aa6d4: done: 0.0152 sec","job_status":"done","queue_duration_s":1.315604,"scheduling_latency_s":1.315621,"redis_calls":13,"redis_duration_s":0.001784,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000566,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001218,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01036,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0152,"completed_at":"2026-10-02T16:56:14.697Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1322gitlab # [ 221.011978] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.699Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24e5e09fa105e2074178ff7c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24e5e09fa105e2074178ff7c: start","job_status":"start","queue_duration_s":1.331967,"scheduling_latency_s":1.331984}1323gitlab # [ 221.025043] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.714Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"24e5e09fa105e2074178ff7c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-24e5e09fa105e2074178ff7c: done: 0.015114 sec","job_status":"done","queue_duration_s":1.331967,"scheduling_latency_s":1.331984,"redis_calls":13,"redis_duration_s":0.001603,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000583,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010493,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015114,"completed_at":"2026-10-02T16:56:14.714Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1324gitlab # [ 221.067534] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.715Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76a04ea79943a60773d89010","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76a04ea79943a60773d89010: start","job_status":"start","queue_duration_s":1.348249,"scheduling_latency_s":1.348267}1325gitlab # [ 221.080686] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.730Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"76a04ea79943a60773d89010","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-76a04ea79943a60773d89010: done: 0.014906 sec","job_status":"done","queue_duration_s":1.348249,"scheduling_latency_s":1.348267,"redis_calls":13,"redis_duration_s":0.001507,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000555,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010054,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014906,"completed_at":"2026-10-02T16:56:14.730Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1326gitlab # [ 221.121309] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1cc4c55fca78f40387f8b9c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1cc4c55fca78f40387f8b9c: start","job_status":"start","queue_duration_s":1.364539,"scheduling_latency_s":1.364558}1327gitlab # [ 221.134249] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.746Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c1cc4c55fca78f40387f8b9c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c1cc4c55fca78f40387f8b9c: done: 0.014923 sec","job_status":"done","queue_duration_s":1.364539,"scheduling_latency_s":1.364558,"redis_calls":13,"redis_duration_s":0.0015899999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001021,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014923,"completed_at":"2026-10-02T16:56:14.746Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1328gitlab # [ 221.175005] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6966920f5ced18b7a342f2","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6966920f5ced18b7a342f2: start","job_status":"start","queue_duration_s":1.380653,"scheduling_latency_s":1.380671}1329gitlab # [ 221.188074] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.762Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d6966920f5ced18b7a342f2","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d6966920f5ced18b7a342f2: done: 0.015247 sec","job_status":"done","queue_duration_s":1.380653,"scheduling_latency_s":1.380671,"redis_calls":13,"redis_duration_s":0.001608,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000572,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001036,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010623,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015247,"completed_at":"2026-10-02T16:56:14.762Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1330gitlab # [ 221.228666] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.764Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3058721dde63525326c2564e","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3058721dde63525326c2564e: start","job_status":"start","queue_duration_s":1.397077,"scheduling_latency_s":1.397096}1331gitlab # [ 221.241518] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.779Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3058721dde63525326c2564e","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3058721dde63525326c2564e: done: 0.015377 sec","job_status":"done","queue_duration_s":1.397077,"scheduling_latency_s":1.397096,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001025,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010696,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015377,"completed_at":"2026-10-02T16:56:14.779Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1332gitlab # [ 221.283793] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.780Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a054858e161d340cea34fd3f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a054858e161d340cea34fd3f: start","job_status":"start","queue_duration_s":1.413618,"scheduling_latency_s":1.413653}1333gitlab # [ 221.304135] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.795Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a054858e161d340cea34fd3f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a054858e161d340cea34fd3f: done: 0.015025 sec","job_status":"done","queue_duration_s":1.413618,"scheduling_latency_s":1.413653,"redis_calls":13,"redis_duration_s":0.001742,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000754,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000988,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010178,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015025,"completed_at":"2026-10-02T16:56:14.795Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1334gitlab # [ 221.362968] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.797Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31922ccc8fbae29416b51d1d","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31922ccc8fbae29416b51d1d: start","job_status":"start","queue_duration_s":1.430185,"scheduling_latency_s":1.430203}1335gitlab # [ 221.377632] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.811Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"31922ccc8fbae29416b51d1d","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-31922ccc8fbae29416b51d1d: done: 0.014786 sec","job_status":"done","queue_duration_s":1.430185,"scheduling_latency_s":1.430203,"redis_calls":13,"redis_duration_s":0.001531,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000969,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014786,"completed_at":"2026-10-02T16:56:14.811Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1336gitlab # [ 221.418595] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.813Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b3e9a43edc0b44a4b5873c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b3e9a43edc0b44a4b5873c: start","job_status":"start","queue_duration_s":1.446208,"scheduling_latency_s":1.446225}1337gitlab # [ 221.431501] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.828Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c0b3e9a43edc0b44a4b5873c","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c0b3e9a43edc0b44a4b5873c: done: 0.015112 sec","job_status":"done","queue_duration_s":1.446208,"scheduling_latency_s":1.446225,"redis_calls":13,"redis_duration_s":0.001587,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00059,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000997,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010529,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015112,"completed_at":"2026-10-02T16:56:14.828Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1338gitlab # [ 221.473348] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.829Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806f140047bf9995466bf7e6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806f140047bf9995466bf7e6: start","job_status":"start","queue_duration_s":1.462573,"scheduling_latency_s":1.46259}1339gitlab # [ 221.486178] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.844Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"806f140047bf9995466bf7e6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-806f140047bf9995466bf7e6: done: 0.014792 sec","job_status":"done","queue_duration_s":1.462573,"scheduling_latency_s":1.46259,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000952,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010285,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014792,"completed_at":"2026-10-02T16:56:14.844Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1340gitlab # [ 221.526624] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.845Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3cf234303a40a1962c1068","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3cf234303a40a1962c1068: start","job_status":"start","queue_duration_s":1.478558,"scheduling_latency_s":1.478576}1341gitlab # [ 221.539522] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.861Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1a3cf234303a40a1962c1068","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1a3cf234303a40a1962c1068: done: 0.015448 sec","job_status":"done","queue_duration_s":1.478558,"scheduling_latency_s":1.478576,"redis_calls":13,"redis_duration_s":0.00149,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010066,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015448,"completed_at":"2026-10-02T16:56:14.860Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1342gitlab # [ 221.580181] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.862Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880a44e17d74aec273381ef0","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880a44e17d74aec273381ef0: start","job_status":"start","queue_duration_s":1.495185,"scheduling_latency_s":1.495203}1343gitlab # [ 221.592973] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.878Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"880a44e17d74aec273381ef0","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-880a44e17d74aec273381ef0: done: 0.016603 sec","job_status":"done","queue_duration_s":1.495185,"scheduling_latency_s":1.495203,"redis_calls":13,"redis_duration_s":0.001549,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000975,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011097,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016603,"completed_at":"2026-10-02T16:56:14.878Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1344gitlab # [ 221.633244] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.880Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fe4200829cdc4c61ccbe9c3","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fe4200829cdc4c61ccbe9c3: start","job_status":"start","queue_duration_s":1.512964,"scheduling_latency_s":1.512981}1345gitlab # [ 221.646008] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.896Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fe4200829cdc4c61ccbe9c3","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fe4200829cdc4c61ccbe9c3: done: 0.016654 sec","job_status":"done","queue_duration_s":1.512964,"scheduling_latency_s":1.512981,"redis_calls":13,"redis_duration_s":0.001843,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000631,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001212,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016654,"completed_at":"2026-10-02T16:56:14.896Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1346gitlab # [ 221.687446] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27fccbc2d4e679553c2c4791","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27fccbc2d4e679553c2c4791: start","job_status":"start","queue_duration_s":1.531261,"scheduling_latency_s":1.53128}1347gitlab # [ 221.700410] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"27fccbc2d4e679553c2c4791","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-27fccbc2d4e679553c2c4791: done: 0.016111 sec","job_status":"done","queue_duration_s":1.531261,"scheduling_latency_s":1.53128,"redis_calls":13,"redis_duration_s":0.001595,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000584,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001011,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011084,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016111,"completed_at":"2026-10-02T16:56:14.914Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1348gitlab # [ 221.740943] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.915Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99567e1b6ed79724331d9ac6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99567e1b6ed79724331d9ac6: start","job_status":"start","queue_duration_s":1.548876,"scheduling_latency_s":1.548894}1349gitlab # [ 221.753746] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.931Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"99567e1b6ed79724331d9ac6","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-99567e1b6ed79724331d9ac6: done: 0.015724 sec","job_status":"done","queue_duration_s":1.548876,"scheduling_latency_s":1.548894,"redis_calls":13,"redis_duration_s":0.001612,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000577,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001035,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011241,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015724,"completed_at":"2026-10-02T16:56:14.931Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1350gitlab # [ 221.794261] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.933Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1aeaeec918c33d6de7d97db","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1aeaeec918c33d6de7d97db: start","job_status":"start","queue_duration_s":1.566181,"scheduling_latency_s":1.5662}1351gitlab # [ 221.807171] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e1aeaeec918c33d6de7d97db","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e1aeaeec918c33d6de7d97db: done: 0.028402 sec","job_status":"done","queue_duration_s":1.566181,"scheduling_latency_s":1.5662,"redis_calls":13,"redis_duration_s":0.006893,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000853,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.028402,"completed_at":"2026-10-02T16:56:14.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1352gitlab # [ 221.848207] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.963Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be153f8113b0f97a3fc22ea","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be153f8113b0f97a3fc22ea: start","job_status":"start","queue_duration_s":1.596134,"scheduling_latency_s":1.596154}1353gitlab # [ 221.868387] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.979Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"4be153f8113b0f97a3fc22ea","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-4be153f8113b0f97a3fc22ea: done: 0.016597 sec","job_status":"done","queue_duration_s":1.596134,"scheduling_latency_s":1.596154,"redis_calls":13,"redis_duration_s":0.002232,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001188,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011342,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016597,"completed_at":"2026-10-02T16:56:14.979Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1354gitlab # [ 221.928814] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.981Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0239611c15d51c4007ce102","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0239611c15d51c4007ce102: start","job_status":"start","queue_duration_s":1.614256,"scheduling_latency_s":1.614276}1355gitlab # [ 221.943618] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.997Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b0239611c15d51c4007ce102","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b0239611c15d51c4007ce102: done: 0.016308 sec","job_status":"done","queue_duration_s":1.614256,"scheduling_latency_s":1.614276,"redis_calls":13,"redis_duration_s":0.0017950000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000586,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001209,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011495,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016308,"completed_at":"2026-10-02T16:56:14.997Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1356gitlab # [ 221.985168] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:14.999Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b583b27b9a19a8a2616d81b","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b583b27b9a19a8a2616d81b: start","job_status":"start","queue_duration_s":1.632598,"scheduling_latency_s":1.632626}1357gitlab # [ 221.998039] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.022Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9b583b27b9a19a8a2616d81b","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9b583b27b9a19a8a2616d81b: done: 0.022624 sec","job_status":"done","queue_duration_s":1.632598,"scheduling_latency_s":1.632626,"redis_calls":13,"redis_duration_s":0.002424,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00082,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001604,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01563,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022624,"completed_at":"2026-10-02T16:56:15.022Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1358gitlab # [ 222.038806] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c95c04f64cf135b33489539d","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c95c04f64cf135b33489539d: start","job_status":"start","queue_duration_s":1.656987,"scheduling_latency_s":1.657014}1359gitlab # [ 222.051867] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.046Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c95c04f64cf135b33489539d","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c95c04f64cf135b33489539d: done: 0.022491 sec","job_status":"done","queue_duration_s":1.656987,"scheduling_latency_s":1.657014,"redis_calls":13,"redis_duration_s":0.002316,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000815,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001501,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015489,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.022491,"completed_at":"2026-10-02T16:56:15.046Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1360gitlab # [ 222.093876] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.048Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef36053fce63ecd219131fa","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef36053fce63ecd219131fa: start","job_status":"start","queue_duration_s":1.681299,"scheduling_latency_s":1.68133}1361gitlab # [ 222.106932] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.071Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3ef36053fce63ecd219131fa","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3ef36053fce63ecd219131fa: done: 0.023022 sec","job_status":"done","queue_duration_s":1.681299,"scheduling_latency_s":1.68133,"redis_calls":13,"redis_duration_s":0.003722,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001156,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002566,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014311,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.023022,"completed_at":"2026-10-02T16:56:15.071Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1362gitlab # [ 222.148054] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1535b5fa2d1c9bc87f8b5ef2","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1535b5fa2d1c9bc87f8b5ef2: start","job_status":"start","queue_duration_s":1.705743,"scheduling_latency_s":1.705763}1363gitlab # [ 222.160836] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.091Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1535b5fa2d1c9bc87f8b5ef2","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1535b5fa2d1c9bc87f8b5ef2: done: 0.018783 sec","job_status":"done","queue_duration_s":1.705743,"scheduling_latency_s":1.705763,"redis_calls":13,"redis_duration_s":0.002137,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000774,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001363,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01242,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018783,"completed_at":"2026-10-02T16:56:15.091Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1364gitlab # [ 222.201191] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.092Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76dfc959146b1e235808800","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76dfc959146b1e235808800: start","job_status":"start","queue_duration_s":1.725822,"scheduling_latency_s":1.725841}1365gitlab # [ 222.213893] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.110Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c76dfc959146b1e235808800","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c76dfc959146b1e235808800: done: 0.017842 sec","job_status":"done","queue_duration_s":1.725822,"scheduling_latency_s":1.725841,"redis_calls":13,"redis_duration_s":0.002112,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001344,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011911,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017842,"completed_at":"2026-10-02T16:56:15.110Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1366gitlab # [ 222.254448] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.112Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aea32bef2aaa98b242899add","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aea32bef2aaa98b242899add: start","job_status":"start","queue_duration_s":1.745311,"scheduling_latency_s":1.74533}1367gitlab # [ 222.267264] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.130Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"aea32bef2aaa98b242899add","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-aea32bef2aaa98b242899add: done: 0.017954 sec","job_status":"done","queue_duration_s":1.745311,"scheduling_latency_s":1.74533,"redis_calls":13,"redis_duration_s":0.0030470000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001782,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001265,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011295,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017954,"completed_at":"2026-10-02T16:56:15.130Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1368gitlab # [ 222.309176] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.131Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"356ed91ea2312fc4d42b8728","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-356ed91ea2312fc4d42b8728: start","job_status":"start","queue_duration_s":1.764884,"scheduling_latency_s":1.764901}1369gitlab # [ 222.322262] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.148Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"356ed91ea2312fc4d42b8728","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-356ed91ea2312fc4d42b8728: done: 0.016186 sec","job_status":"done","queue_duration_s":1.764884,"scheduling_latency_s":1.764901,"redis_calls":13,"redis_duration_s":0.0018930000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000664,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001229,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011306,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016186,"completed_at":"2026-10-02T16:56:15.148Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1370gitlab # [ 222.362890] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.149Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a506e07a8f340e6746449f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a506e07a8f340e6746449f: start","job_status":"start","queue_duration_s":1.782711,"scheduling_latency_s":1.782729}1371gitlab # [ 222.375974] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.166Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"25a506e07a8f340e6746449f","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-25a506e07a8f340e6746449f: done: 0.01685 sec","job_status":"done","queue_duration_s":1.782711,"scheduling_latency_s":1.782729,"redis_calls":13,"redis_duration_s":0.002032,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000701,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001331,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011737,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01685,"completed_at":"2026-10-02T16:56:15.166Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1372gitlab # [ 222.416720] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f63385c6d21084f989917c66","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f63385c6d21084f989917c66: start","job_status":"start","queue_duration_s":1.801264,"scheduling_latency_s":1.801282}1373gitlab # [ 222.429534] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.187Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f63385c6d21084f989917c66","created_at":"2026-10-02T16:56:13.281Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-81edca171d384270","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"17a82df42e0eccc160ae05d2a73966e9","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.367Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f63385c6d21084f989917c66: done: 0.018813 sec","job_status":"done","queue_duration_s":1.801264,"scheduling_latency_s":1.801282,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001303,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013999,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018813,"completed_at":"2026-10-02T16:56:15.187Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1374gitlab # [ 222.470067] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.188Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eab9b4a67c8d4de16b2ea30b","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eab9b4a67c8d4de16b2ea30b: start","job_status":"start","queue_duration_s":1.358063,"scheduling_latency_s":1.358081}1375gitlab # [ 222.483856] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.208Z","message":"No existing merge request to be cleaned up."}1376gitlab # [ 222.485677] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.225Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"eab9b4a67c8d4de16b2ea30b","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-eab9b4a67c8d4de16b2ea30b: done: 0.036883 sec","job_status":"done","queue_duration_s":1.358063,"scheduling_latency_s":1.358081,"redis_calls":13,"redis_duration_s":0.0026709999999999998,"redis_read_bytes":12,"redis_write_bytes":1401,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000412,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001189,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":431,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00107,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023789,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029578,"duration_s":0.036883,"completed_at":"2026-10-02T16:56:15.225Z","load_balancing_strategy":"primary","db_duration_s":0.004396,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1377gitlab # [ 222.527198] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.226Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acaca6edaa0735fb757a3b51","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acaca6edaa0735fb757a3b51: start","job_status":"start","queue_duration_s":1.396285,"scheduling_latency_s":1.396302}1378gitlab # [ 222.540138] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.238Z","message":"No existing merge request to be cleaned up."}1379gitlab # [ 222.541841] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.248Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"acaca6edaa0735fb757a3b51","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-acaca6edaa0735fb757a3b51: done: 0.021308 sec","job_status":"done","queue_duration_s":1.396285,"scheduling_latency_s":1.396302,"redis_calls":10,"redis_duration_s":0.00181,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001017,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014769,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021308,"completed_at":"2026-10-02T16:56:15.248Z","load_balancing_strategy":"primary","db_duration_s":0.001836,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1380gitlab # [ 222.581601] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.250Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d344e60d4ab25b13bcb8b7eb","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d344e60d4ab25b13bcb8b7eb: start","job_status":"start","queue_duration_s":1.419382,"scheduling_latency_s":1.419401}1381gitlab # [ 222.594246] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.261Z","message":"No existing merge request to be cleaned up."}1382gitlab # [ 222.596009] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.269Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"d344e60d4ab25b13bcb8b7eb","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-d344e60d4ab25b13bcb8b7eb: done: 0.019872 sec","job_status":"done","queue_duration_s":1.419382,"scheduling_latency_s":1.419401,"redis_calls":10,"redis_duration_s":0.0017239999999999998,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000723,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001001,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013995,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019872,"completed_at":"2026-10-02T16:56:15.269Z","load_balancing_strategy":"primary","db_duration_s":0.001675,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1383gitlab # [ 222.636204] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.271Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdc18b6a91df9828c03d8a76","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdc18b6a91df9828c03d8a76: start","job_status":"start","queue_duration_s":1.440483,"scheduling_latency_s":1.440499}1384gitlab # [ 222.648749] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.281Z","message":"No existing merge request to be cleaned up."}1385gitlab # [ 222.650463] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:56:15.289Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"cdc18b6a91df9828c03d8a76","created_at":"2026-10-02T16:56:13.816Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-6f9d99d49e7e4314","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"67bd5d3f46fb29fccadd9c6edbea2d56","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:56:13.830Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-cdc18b6a91df9828c03d8a76: done: 0.018616 sec","job_status":"done","queue_duration_s":1.440483,"scheduling_latency_s":1.440499,"redis_calls":10,"redis_duration_s":0.001804,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00076,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001044,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012717,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018616,"completed_at":"2026-10-02T16:56:15.289Z","load_balancing_strategy":"primary","db_duration_s":0.00172,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1386gitlab # [ 258.566319] setup-gitlab-start[2894]: W, [2026-10-02T16:56:53.502233 #2894] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1387gitlab # [ 266.799416] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:01.734Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1808100d0e7c92dd94d56b3c","created_at":"2026-10-02T16:57:01.714Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85a8f87e09e98517e18583a1b3adc940","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.729Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1808100d0e7c92dd94d56b3c: start","job_status":"start","queue_duration_s":0.005723,"scheduling_latency_s":0.005765}1388gitlab # [ 267.182573] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.115Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1808100d0e7c92dd94d56b3c","created_at":"2026-10-02T16:57:01.714Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"85a8f87e09e98517e18583a1b3adc940","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.729Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1808100d0e7c92dd94d56b3c: done: 0.381018 sec","job_status":"done","queue_duration_s":0.005723,"scheduling_latency_s":0.005765,"redis_calls":17,"redis_duration_s":0.053959999999999994,"redis_read_bytes":1031,"redis_write_bytes":1585,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.018347,"redis_feature_flag_read_bytes":1024,"redis_feature_flag_write_bytes":513,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012256,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.023357,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.031,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030576,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00722824,"exclusive_lock_hold_duration_s":0.03830152,"duration_s":0.381018,"completed_at":"2026-10-02T16:57:02.115Z","load_balancing_strategy":"primary","db_duration_s":0.030524,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1389gitlab # [ 267.241804] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.131Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3d23ce1fa4c45e497bc8981b","created_at":"2026-10-02T16:57:01.797Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a34557f7f21dccdd167a52dff53d4e4c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.806Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3d23ce1fa4c45e497bc8981b: start","job_status":"start","queue_duration_s":0.325372,"scheduling_latency_s":0.325412}1390gitlab # [ 267.455119] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.388Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"3d23ce1fa4c45e497bc8981b","created_at":"2026-10-02T16:57:01.797Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a34557f7f21dccdd167a52dff53d4e4c","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.806Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-3d23ce1fa4c45e497bc8981b: done: 0.256992 sec","job_status":"done","queue_duration_s":0.325372,"scheduling_latency_s":0.325412,"redis_calls":9,"redis_duration_s":0.03633,"redis_read_bytes":7,"redis_write_bytes":1087,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.024678,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.011652,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018712,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00270776,"exclusive_lock_hold_duration_s":0.01162768,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.256992,"completed_at":"2026-10-02T16:57:02.388Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.008562,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1391gitlab # [ 267.524033] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.460Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e3be8b6e8353495b07af7cf9","created_at":"2026-10-02T16:57:01.823Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c4b989b607518f355e5377cf46901a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.889Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e3be8b6e8353495b07af7cf9: start","job_status":"start","queue_duration_s":0.570842,"scheduling_latency_s":0.570895}1392gitlab # [ 267.658307] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.592Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"e3be8b6e8353495b07af7cf9","created_at":"2026-10-02T16:57:01.823Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4c4b989b607518f355e5377cf46901a9","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.889Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-e3be8b6e8353495b07af7cf9: done: 0.132764 sec","job_status":"done","queue_duration_s":0.570842,"scheduling_latency_s":0.570895,"redis_calls":8,"redis_duration_s":0.014635,"redis_read_bytes":133,"redis_write_bytes":901,"redis_feature_flag_calls":1,"redis_feature_flag_duration_s":0.002892,"redis_feature_flag_read_bytes":128,"redis_feature_flag_write_bytes":70,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006727,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.005016,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016724,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.132764,"completed_at":"2026-10-02T16:57:02.592Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1393gitlab # [ 267.724304] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.660Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fce875e5a1a3b983471a0520","created_at":"2026-10-02T16:57:01.908Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8a02f3536b5692c7d569aa7df08bde3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.915Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fce875e5a1a3b983471a0520: start","job_status":"start","queue_duration_s":0.744984,"scheduling_latency_s":0.745015}1394gitlab # [ 267.779192] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.715Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}1395gitlab # [ 267.805844] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.741Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"fce875e5a1a3b983471a0520","created_at":"2026-10-02T16:57:01.908Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c8a02f3536b5692c7d569aa7df08bde3","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.915Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-fce875e5a1a3b983471a0520: done: 0.080835 sec","job_status":"done","queue_duration_s":0.744984,"scheduling_latency_s":0.745015,"redis_calls":7,"redis_duration_s":0.008566,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005928,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002638,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014286,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.080835,"completed_at":"2026-10-02T16:57:02.741Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1396gitlab # [ 267.866098] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.748Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8da279e5904c326fa720ca99","created_at":"2026-10-02T16:57:01.979Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02a3d2827ec2c43086a253ad5a8fe405","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.990Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8da279e5904c326fa720ca99: start","job_status":"start","queue_duration_s":0.758654,"scheduling_latency_s":0.758685}1397gitlab # [ 267.882617] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.804Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"8da279e5904c326fa720ca99","created_at":"2026-10-02T16:57:01.979Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"02a3d2827ec2c43086a253ad5a8fe405","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:01.990Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-8da279e5904c326fa720ca99: done: 0.055769 sec","job_status":"done","queue_duration_s":0.758654,"scheduling_latency_s":0.758685,"redis_calls":7,"redis_duration_s":0.009861,"redis_read_bytes":5,"redis_write_bytes":776,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007132,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002729,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015959,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.055769,"completed_at":"2026-10-02T16:57:02.804Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1398gitlab # [ 267.924877] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:02.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"04645156d339ec0fab1d0849","created_at":"2026-10-02T16:57:02.010Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9a6d1a2b0800381fc334bbdce2cd1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.025Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-04645156d339ec0fab1d0849: start","job_status":"start","queue_duration_s":0.785965,"scheduling_latency_s":0.785999}1399gitlab # [ 269.094152] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:04.029Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"04645156d339ec0fab1d0849","created_at":"2026-10-02T16:57:02.010Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"d9a6d1a2b0800381fc334bbdce2cd1b3","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.025Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-04645156d339ec0fab1d0849: done: 1.218163 sec","job_status":"done","queue_duration_s":0.785965,"scheduling_latency_s":0.785999,"redis_calls":562,"redis_duration_s":0.744634,"redis_read_bytes":560,"redis_write_bytes":48760,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.008101,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.736533,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.5577,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":1.218163,"completed_at":"2026-10-02T16:57:04.029Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1400gitlab # [ 269.157906] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:04.032Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"da8be09b22b57065f0a70720","created_at":"2026-10-02T16:57:02.047Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.054Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-da8be09b22b57065f0a70720: start","job_status":"start","queue_duration_s":1.977598,"scheduling_latency_s":1.977638}1401gitlab # [ 269.420841] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:04.356Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"da8be09b22b57065f0a70720","created_at":"2026-10-02T16:57:02.047Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.054Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-da8be09b22b57065f0a70720: done: 0.3243 sec","job_status":"done","queue_duration_s":1.977598,"scheduling_latency_s":1.977638,"redis_calls":60,"redis_duration_s":0.081206,"redis_read_bytes":108,"redis_write_bytes":41109,"redis_queues_calls":2,"redis_queues_duration_s":0.002331,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.077611,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001264,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.147721,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.3243,"completed_at":"2026-10-02T16:57:04.356Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1402gitlab # [ 269.496238] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:04.360Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"886834f2bbb18fb0bd7675b9","created_at":"2026-10-02T16:57:02.084Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b37e60a80c86bcdf2f924f49eb1ae27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.098Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-886834f2bbb18fb0bd7675b9: start","job_status":"start","queue_duration_s":2.261239,"scheduling_latency_s":2.261275}1403gitlab # [ 270.050153] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:04.984Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"886834f2bbb18fb0bd7675b9","created_at":"2026-10-02T16:57:02.084Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9b37e60a80c86bcdf2f924f49eb1ae27","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.098Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-886834f2bbb18fb0bd7675b9: done: 0.624864 sec","job_status":"done","queue_duration_s":2.261239,"scheduling_latency_s":2.261275,"redis_calls":9,"redis_duration_s":0.007488999999999999,"redis_read_bytes":7,"redis_write_bytes":987,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.003133,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.004356,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":129,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":129,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.132,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.132,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.34119,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00100718,"exclusive_lock_hold_duration_s":0.583143,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.624864,"completed_at":"2026-10-02T16:57:04.984Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.295221,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1404gitlab # [ 270.121571] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"85bc30c896e3db95e3efa0f8","created_at":"2026-10-02T16:57:02.113Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8306788f11b99be5c3cf60b1468d1f3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.126Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-85bc30c896e3db95e3efa0f8: start","job_status":"start","queue_duration_s":2.930019,"scheduling_latency_s":2.930052}1405gitlab # [ 270.255639] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.191Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"85bc30c896e3db95e3efa0f8","created_at":"2026-10-02T16:57:02.113Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8306788f11b99be5c3cf60b1468d1f3e","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.126Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-85bc30c896e3db95e3efa0f8: done: 0.135575 sec","job_status":"done","queue_duration_s":2.930019,"scheduling_latency_s":2.930052,"redis_calls":9,"redis_duration_s":0.00737,"redis_read_bytes":7,"redis_write_bytes":1169,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005285,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.002085,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.039,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.039,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032819,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00141454,"exclusive_lock_hold_duration_s":0.02984492,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.135575,"completed_at":"2026-10-02T16:57:05.191Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.046534,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1406gitlab # [ 270.317872] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.193Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0e576ca59a35b2f09996fded","created_at":"2026-10-02T16:57:02.144Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a37e78f8e207c7e282528437a9446bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.152Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0e576ca59a35b2f09996fded: start","job_status":"start","queue_duration_s":3.041323,"scheduling_latency_s":3.041349}1407gitlab # [ 270.333435] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.214Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"0e576ca59a35b2f09996fded","created_at":"2026-10-02T16:57:02.144Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a37e78f8e207c7e282528437a9446bc3","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.152Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-0e576ca59a35b2f09996fded: done: 0.020995 sec","job_status":"done","queue_duration_s":3.041323,"scheduling_latency_s":3.041349,"redis_calls":7,"redis_duration_s":0.001428,"redis_read_bytes":5,"redis_write_bytes":681,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001031,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000397,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020995,"completed_at":"2026-10-02T16:57:05.214Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0011,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1408gitlab # [ 270.375431] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.215Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f8c5dbc7b5b00a69442ec603","created_at":"2026-10-02T16:57:02.156Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41fbf9c223b710f91146a6fc3acb9fd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.164Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f8c5dbc7b5b00a69442ec603: start","job_status":"start","queue_duration_s":3.051505,"scheduling_latency_s":3.051527}1409gitlab # [ 270.390635] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.225Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}1410gitlab # [ 270.394466] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.231Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"f8c5dbc7b5b00a69442ec603","created_at":"2026-10-02T16:57:02.156Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"41fbf9c223b710f91146a6fc3acb9fd6","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.164Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-f8c5dbc7b5b00a69442ec603: done: 0.015557 sec","job_status":"done","queue_duration_s":3.051505,"scheduling_latency_s":3.051527,"redis_calls":7,"redis_duration_s":0.001344,"redis_read_bytes":5,"redis_write_bytes":766,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000968,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000376,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.008754,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015557,"completed_at":"2026-10-02T16:57:05.231Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1411gitlab # [ 270.438261] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.234Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3d7a6590fd5b8981f6da712a","created_at":"2026-10-02T16:57:02.168Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74df3212afa03db5408a8a7bcb6099db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.174Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3d7a6590fd5b8981f6da712a: start","job_status":"start","queue_duration_s":3.05957,"scheduling_latency_s":3.059589}1412gitlab # [ 270.453529] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.242Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}1413gitlab # [ 270.457020] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.248Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"3d7a6590fd5b8981f6da712a","created_at":"2026-10-02T16:57:02.168Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"74df3212afa03db5408a8a7bcb6099db","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.174Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-3d7a6590fd5b8981f6da712a: done: 0.014634 sec","job_status":"done","queue_duration_s":3.05957,"scheduling_latency_s":3.059589,"redis_calls":7,"redis_duration_s":0.001596,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009512,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014634,"completed_at":"2026-10-02T16:57:05.248Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1414gitlab # [ 270.499262] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.249Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9e1ddfda5c26a818caf0d9c3","created_at":"2026-10-02T16:57:02.187Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.192Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9e1ddfda5c26a818caf0d9c3: start","job_status":"start","queue_duration_s":3.057198,"scheduling_latency_s":3.057215}1415gitlab # [ 270.513237] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.278Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"9e1ddfda5c26a818caf0d9c3","created_at":"2026-10-02T16:57:02.187Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.192Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-9e1ddfda5c26a818caf0d9c3: done: 0.028151 sec","job_status":"done","queue_duration_s":3.057198,"scheduling_latency_s":3.057215,"redis_calls":14,"redis_duration_s":0.002431,"redis_read_bytes":16,"redis_write_bytes":4030,"redis_queues_calls":2,"redis_queues_duration_s":0.000275,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3019,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001609,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000547,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028151,"completed_at":"2026-10-02T16:57:05.278Z","load_balancing_strategy":"primary","db_duration_s":0.001182,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1416gitlab # [ 270.555968] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.279Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f2721942f72d00213c35b68","created_at":"2026-10-02T16:57:02.197Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e5830485bb8984f4c9d1cd82df880c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:57:02.203Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f2721942f72d00213c35b68: start","job_status":"start","queue_duration_s":3.075847,"scheduling_latency_s":3.075864}1417gitlab # [ 270.569913] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.288Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}1418gitlab # [ 270.573772] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.293Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"3f2721942f72d00213c35b68","created_at":"2026-10-02T16:57:02.197Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1e5830485bb8984f4c9d1cd82df880c1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:57:02.203Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-3f2721942f72d00213c35b68: done: 0.014019 sec","job_status":"done","queue_duration_s":3.075847,"scheduling_latency_s":3.075864,"redis_calls":7,"redis_duration_s":0.00123,"redis_read_bytes":5,"redis_write_bytes":801,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000884,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000346,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.00801,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.014019,"completed_at":"2026-10-02T16:57:05.293Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1419gitlab # [ 270.614926] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.295Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a9e1cf8efd3bfbf46788726e","created_at":"2026-10-02T16:57:02.210Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82d8f0cd78af8ad6207674b84535aee2","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.217Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a9e1cf8efd3bfbf46788726e: start","job_status":"start","queue_duration_s":3.078123,"scheduling_latency_s":3.07814}1420gitlab # [ 270.630517] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.318Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"a9e1cf8efd3bfbf46788726e","created_at":"2026-10-02T16:57:02.210Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"82d8f0cd78af8ad6207674b84535aee2","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.217Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-a9e1cf8efd3bfbf46788726e: done: 0.022928 sec","job_status":"done","queue_duration_s":3.078123,"scheduling_latency_s":3.07814,"redis_calls":7,"redis_duration_s":0.001254,"redis_read_bytes":5,"redis_write_bytes":826,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000919,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000335,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.022928,"completed_at":"2026-10-02T16:57:05.318Z","load_balancing_strategy":"primary","db_duration_s":0.007414,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1421gitlab # [ 270.673392] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.320Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1df73e1a176930bdd8f9f291","created_at":"2026-10-02T16:57:02.239Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab55948fc60d8393f1cd5866f5eb72a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:57:02.250Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1df73e1a176930bdd8f9f291: start","job_status":"start","queue_duration_s":3.069757,"scheduling_latency_s":3.069774}1422gitlab # [ 270.687274] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.329Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}1423gitlab # [ 270.690909] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.335Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"1df73e1a176930bdd8f9f291","created_at":"2026-10-02T16:57:02.239Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ab55948fc60d8393f1cd5866f5eb72a1","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:57:02.250Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-1df73e1a176930bdd8f9f291: done: 0.015364 sec","job_status":"done","queue_duration_s":3.069757,"scheduling_latency_s":3.069774,"redis_calls":7,"redis_duration_s":0.001343,"redis_read_bytes":5,"redis_write_bytes":806,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000957,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000386,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010246,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.015364,"completed_at":"2026-10-02T16:57:05.335Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1424gitlab # [ 270.732315] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.336Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"009c5133cc046848cf1afc67","created_at":"2026-10-02T16:57:02.276Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"685573d81ed60a51c6358ba7732ef68f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.284Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-009c5133cc046848cf1afc67: start","job_status":"start","queue_duration_s":3.052002,"scheduling_latency_s":3.05204}1425gitlab # [ 270.747277] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.382Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"009c5133cc046848cf1afc67","created_at":"2026-10-02T16:57:02.276Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"685573d81ed60a51c6358ba7732ef68f","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.284Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-009c5133cc046848cf1afc67: done: 0.045447 sec","job_status":"done","queue_duration_s":3.052002,"scheduling_latency_s":3.05204,"redis_calls":9,"redis_duration_s":0.0017360000000000001,"redis_read_bytes":7,"redis_write_bytes":1162,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000954,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000782,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":6,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023631,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00052566,"exclusive_lock_hold_duration_s":0.02807288,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.045447,"completed_at":"2026-10-02T16:57:05.382Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016609,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1426gitlab # [ 270.792856] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.383Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"18b905911ff3472cce889a15","created_at":"2026-10-02T16:57:02.304Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f85cddd3ff8211b99801056611a3c7ac","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.320Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-18b905911ff3472cce889a15: start","job_status":"start","queue_duration_s":3.063169,"scheduling_latency_s":3.063186}1427gitlab # [ 270.807810] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.404Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"18b905911ff3472cce889a15","created_at":"2026-10-02T16:57:02.304Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f85cddd3ff8211b99801056611a3c7ac","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.320Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-18b905911ff3472cce889a15: done: 0.021051 sec","job_status":"done","queue_duration_s":3.063169,"scheduling_latency_s":3.063186,"redis_calls":10,"redis_duration_s":0.00154,"redis_read_bytes":8,"redis_write_bytes":1239,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000917,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000623,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014675,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00609488,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.021051,"completed_at":"2026-10-02T16:57:05.404Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002181,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1428gitlab # [ 270.853710] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.405Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2c158f018084790d02ec4e79","created_at":"2026-10-02T16:57:02.348Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da1dd05df01cff3c9e4b4314c6b514fe","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.363Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2c158f018084790d02ec4e79: start","job_status":"start","queue_duration_s":3.042427,"scheduling_latency_s":3.042445}1429gitlab # [ 270.868618] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.425Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"2c158f018084790d02ec4e79","created_at":"2026-10-02T16:57:02.348Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"da1dd05df01cff3c9e4b4314c6b514fe","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.363Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-2c158f018084790d02ec4e79: done: 0.019757 sec","job_status":"done","queue_duration_s":3.042427,"scheduling_latency_s":3.042445,"redis_calls":7,"redis_duration_s":0.001052,"redis_read_bytes":5,"redis_write_bytes":711,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000764,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000288,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019757,"completed_at":"2026-10-02T16:57:05.425Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002946,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1430gitlab # [ 270.910356] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.426Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"df3bc3bd9e4992030753f374","created_at":"2026-10-02T16:57:02.381Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0a3d060d0dfb8a8f92870829179b655","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:57:02.391Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-df3bc3bd9e4992030753f374: start","job_status":"start","queue_duration_s":3.035267,"scheduling_latency_s":3.035285}1431gitlab # [ 270.924245] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.453Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"df3bc3bd9e4992030753f374","created_at":"2026-10-02T16:57:02.381Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"c0a3d060d0dfb8a8f92870829179b655","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:57:02.391Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-df3bc3bd9e4992030753f374: done: 0.026763 sec","job_status":"done","queue_duration_s":3.035267,"scheduling_latency_s":3.035285,"redis_calls":7,"redis_duration_s":0.0010630000000000001,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00075,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000313,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019699,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.026763,"completed_at":"2026-10-02T16:57:05.453Z","load_balancing_strategy":"primary","db_duration_s":0.001401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1432gitlab # [ 270.965326] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.454Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"589d5c3db9b7b0042df24587","created_at":"2026-10-02T16:57:02.499Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57cc9cbee1d3b13f64ff242392da6390","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.508Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-589d5c3db9b7b0042df24587: start","job_status":"start","queue_duration_s":2.946432,"scheduling_latency_s":2.946449}1433gitlab # [ 270.980644] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.463Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}1434gitlab # [ 270.984542] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.468Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"589d5c3db9b7b0042df24587","created_at":"2026-10-02T16:57:02.499Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-f039578dd398481b","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"57cc9cbee1d3b13f64ff242392da6390","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:02.508Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-589d5c3db9b7b0042df24587: done: 0.013934 sec","job_status":"done","queue_duration_s":2.946432,"scheduling_latency_s":2.946449,"redis_calls":7,"redis_duration_s":0.001036,"redis_read_bytes":5,"redis_write_bytes":821,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00032,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009028,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.013934,"completed_at":"2026-10-02T16:57:05.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1435gitlab # [ 271.026833] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.469Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47a3def6fa83e7ec19783b75","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47a3def6fa83e7ec19783b75: start","job_status":"start","queue_duration_s":1.135849,"scheduling_latency_s":1.135866}1436gitlab # [ 271.040874] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.485Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"47a3def6fa83e7ec19783b75","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-47a3def6fa83e7ec19783b75: done: 0.016202 sec","job_status":"done","queue_duration_s":1.135849,"scheduling_latency_s":1.135866,"redis_calls":13,"redis_duration_s":0.0016489999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001078,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010386,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016202,"completed_at":"2026-10-02T16:57:05.485Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1437gitlab # [ 271.081870] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.488Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46966e1461d964c67ada8362","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46966e1461d964c67ada8362: start","job_status":"start","queue_duration_s":1.15434,"scheduling_latency_s":1.154357}1438gitlab # [ 271.094745] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.503Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"46966e1461d964c67ada8362","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-46966e1461d964c67ada8362: done: 0.015532 sec","job_status":"done","queue_duration_s":1.15434,"scheduling_latency_s":1.154357,"redis_calls":13,"redis_duration_s":0.001553,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000966,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015532,"completed_at":"2026-10-02T16:57:05.503Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1439gitlab # [ 271.136206] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.504Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d05706f79d32ace25892218b","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d05706f79d32ace25892218b: start","job_status":"start","queue_duration_s":1.171007,"scheduling_latency_s":1.171024}1440gitlab # [ 271.149045] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.520Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d05706f79d32ace25892218b","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d05706f79d32ace25892218b: done: 0.015698 sec","job_status":"done","queue_duration_s":1.171007,"scheduling_latency_s":1.171024,"redis_calls":13,"redis_duration_s":0.0015400000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011236,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015698,"completed_at":"2026-10-02T16:57:05.520Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1441gitlab # [ 271.189753] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.521Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ee8ca29b1b924d29db97c8","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ee8ca29b1b924d29db97c8: start","job_status":"start","queue_duration_s":1.187815,"scheduling_latency_s":1.187833}1442gitlab # [ 271.202806] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.537Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e5ee8ca29b1b924d29db97c8","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e5ee8ca29b1b924d29db97c8: done: 0.015533 sec","job_status":"done","queue_duration_s":1.187815,"scheduling_latency_s":1.187833,"redis_calls":13,"redis_duration_s":0.001522,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000546,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011124,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015533,"completed_at":"2026-10-02T16:57:05.537Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1443gitlab # [ 271.244760] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.538Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69e91cca687c43b3700fa421","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69e91cca687c43b3700fa421: start","job_status":"start","queue_duration_s":1.20444,"scheduling_latency_s":1.204456}1444gitlab # [ 271.257771] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.555Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69e91cca687c43b3700fa421","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69e91cca687c43b3700fa421: done: 0.017115 sec","job_status":"done","queue_duration_s":1.20444,"scheduling_latency_s":1.204456,"redis_calls":13,"redis_duration_s":0.0015600000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000581,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000979,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009978,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017115,"completed_at":"2026-10-02T16:57:05.555Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1445gitlab # [ 271.298773] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.556Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95a78d92c4500e4dc6fa26","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95a78d92c4500e4dc6fa26: start","job_status":"start","queue_duration_s":1.222765,"scheduling_latency_s":1.222782}1446gitlab # [ 271.311852] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.572Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"be95a78d92c4500e4dc6fa26","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-be95a78d92c4500e4dc6fa26: done: 0.015699 sec","job_status":"done","queue_duration_s":1.222765,"scheduling_latency_s":1.222782,"redis_calls":13,"redis_duration_s":0.001614,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000548,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001066,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011228,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015699,"completed_at":"2026-10-02T16:57:05.572Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1447gitlab # [ 271.353156] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.573Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f92f50f9a4ded024ceb5a376","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f92f50f9a4ded024ceb5a376: start","job_status":"start","queue_duration_s":1.23959,"scheduling_latency_s":1.239606}1448gitlab # [ 271.366093] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.588Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f92f50f9a4ded024ceb5a376","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f92f50f9a4ded024ceb5a376: done: 0.015313 sec","job_status":"done","queue_duration_s":1.23959,"scheduling_latency_s":1.239606,"redis_calls":13,"redis_duration_s":0.0014889999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00095,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010992,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015313,"completed_at":"2026-10-02T16:57:05.588Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1449gitlab # [ 271.407369] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.589Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889d0a1ac273187835af010d","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889d0a1ac273187835af010d: start","job_status":"start","queue_duration_s":1.25601,"scheduling_latency_s":1.256026}1450gitlab # [ 271.420448] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"889d0a1ac273187835af010d","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-889d0a1ac273187835af010d: done: 0.015979 sec","job_status":"done","queue_duration_s":1.25601,"scheduling_latency_s":1.256026,"redis_calls":13,"redis_duration_s":0.001588,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000617,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000971,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010588,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015979,"completed_at":"2026-10-02T16:57:05.605Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1451gitlab # [ 271.462483] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.606Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b37c65c44ef24df27aeab6b6","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b37c65c44ef24df27aeab6b6: start","job_status":"start","queue_duration_s":1.273171,"scheduling_latency_s":1.273188}1452gitlab # [ 271.475503] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b37c65c44ef24df27aeab6b6","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b37c65c44ef24df27aeab6b6: done: 0.014877 sec","job_status":"done","queue_duration_s":1.273171,"scheduling_latency_s":1.273188,"redis_calls":13,"redis_duration_s":0.001513,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000951,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010465,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014877,"completed_at":"2026-10-02T16:57:05.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1453gitlab # [ 271.516499] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed6e357d05e4bac41a0b5319","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed6e357d05e4bac41a0b5319: start","job_status":"start","queue_duration_s":1.289143,"scheduling_latency_s":1.28916}1454gitlab # [ 271.529283] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.637Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed6e357d05e4bac41a0b5319","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed6e357d05e4bac41a0b5319: done: 0.014999 sec","job_status":"done","queue_duration_s":1.289143,"scheduling_latency_s":1.28916,"redis_calls":13,"redis_duration_s":0.00155,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001014,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010482,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014999,"completed_at":"2026-10-02T16:57:05.637Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1455gitlab # [ 271.569960] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.638Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1834acd146ac884e80aa69dd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1834acd146ac884e80aa69dd: start","job_status":"start","queue_duration_s":1.30525,"scheduling_latency_s":1.305267}1456gitlab # [ 271.582969] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.654Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1834acd146ac884e80aa69dd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1834acd146ac884e80aa69dd: done: 0.01514 sec","job_status":"done","queue_duration_s":1.30525,"scheduling_latency_s":1.305267,"redis_calls":13,"redis_duration_s":0.0015500000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000551,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000999,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010499,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01514,"completed_at":"2026-10-02T16:57:05.654Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1457gitlab # [ 271.624001] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.655Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abd000a765754742070b32bd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abd000a765754742070b32bd: start","job_status":"start","queue_duration_s":1.322152,"scheduling_latency_s":1.32217}1458gitlab # [ 271.636607] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.670Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"abd000a765754742070b32bd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-abd000a765754742070b32bd: done: 0.015011 sec","job_status":"done","queue_duration_s":1.322152,"scheduling_latency_s":1.32217,"redis_calls":13,"redis_duration_s":0.001552,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00057,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010448,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015011,"completed_at":"2026-10-02T16:57:05.670Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1459gitlab # [ 271.678335] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.674Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"140166b348f24f4a026ce03c","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-140166b348f24f4a026ce03c: start","job_status":"start","queue_duration_s":1.340455,"scheduling_latency_s":1.340474}1460gitlab # [ 271.691270] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.689Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"140166b348f24f4a026ce03c","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-140166b348f24f4a026ce03c: done: 0.014877 sec","job_status":"done","queue_duration_s":1.340455,"scheduling_latency_s":1.340474,"redis_calls":13,"redis_duration_s":0.0015210000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010364,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014877,"completed_at":"2026-10-02T16:57:05.688Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1461gitlab # [ 271.732519] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.690Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8209687a2cfb30f1714aa771","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8209687a2cfb30f1714aa771: start","job_status":"start","queue_duration_s":1.35646,"scheduling_latency_s":1.356478}1462gitlab # [ 271.745350] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.704Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8209687a2cfb30f1714aa771","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8209687a2cfb30f1714aa771: done: 0.014723 sec","job_status":"done","queue_duration_s":1.35646,"scheduling_latency_s":1.356478,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00054,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000942,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010328,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014723,"completed_at":"2026-10-02T16:57:05.704Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1463gitlab # [ 271.785969] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.706Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f42fef9a38d34cf7b50e9923","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f42fef9a38d34cf7b50e9923: start","job_status":"start","queue_duration_s":1.372283,"scheduling_latency_s":1.3723}1464gitlab # [ 271.798837] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.721Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f42fef9a38d34cf7b50e9923","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f42fef9a38d34cf7b50e9923: done: 0.015105 sec","job_status":"done","queue_duration_s":1.372283,"scheduling_latency_s":1.3723,"redis_calls":13,"redis_duration_s":0.0015010000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000937,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009796,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015105,"completed_at":"2026-10-02T16:57:05.721Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1465gitlab # [ 271.839813] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.722Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f89b8cd16cc67242f42a8a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f89b8cd16cc67242f42a8a: start","job_status":"start","queue_duration_s":1.388515,"scheduling_latency_s":1.388533}1466gitlab # [ 271.853804] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.737Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f2f89b8cd16cc67242f42a8a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f2f89b8cd16cc67242f42a8a: done: 0.014858 sec","job_status":"done","queue_duration_s":1.388515,"scheduling_latency_s":1.388533,"redis_calls":13,"redis_duration_s":0.001523,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000561,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000962,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010356,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014858,"completed_at":"2026-10-02T16:57:05.737Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1467gitlab # [ 271.894518] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.738Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2d8e50aa43b54575af5025","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2d8e50aa43b54575af5025: start","job_status":"start","queue_duration_s":1.404467,"scheduling_latency_s":1.404483}1468gitlab # [ 271.907575] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.753Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3e2d8e50aa43b54575af5025","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3e2d8e50aa43b54575af5025: done: 0.015306 sec","job_status":"done","queue_duration_s":1.404467,"scheduling_latency_s":1.404483,"redis_calls":13,"redis_duration_s":0.0018579999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00056,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001298,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010557,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015306,"completed_at":"2026-10-02T16:57:05.753Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1469gitlab # [ 271.948822] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.754Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ee8deef989eeb6091278bd0","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ee8deef989eeb6091278bd0: start","job_status":"start","queue_duration_s":1.420871,"scheduling_latency_s":1.420888}1470gitlab # [ 271.961695] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.769Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2ee8deef989eeb6091278bd0","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2ee8deef989eeb6091278bd0: done: 0.01467 sec","job_status":"done","queue_duration_s":1.420871,"scheduling_latency_s":1.420888,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010273,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01467,"completed_at":"2026-10-02T16:57:05.769Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1471gitlab # [ 272.002321] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.770Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d531b36606aee1067d42ea5","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d531b36606aee1067d42ea5: start","job_status":"start","queue_duration_s":1.436657,"scheduling_latency_s":1.436674}1472gitlab # [ 272.015347] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d531b36606aee1067d42ea5","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d531b36606aee1067d42ea5: done: 0.014998 sec","job_status":"done","queue_duration_s":1.436657,"scheduling_latency_s":1.436674,"redis_calls":13,"redis_duration_s":0.0014810000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000929,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009883,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014998,"completed_at":"2026-10-02T16:57:05.785Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1473gitlab # [ 272.057597] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.786Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb8c7283ce8e2bb9c8f8386f","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb8c7283ce8e2bb9c8f8386f: start","job_status":"start","queue_duration_s":1.452794,"scheduling_latency_s":1.452811}1474gitlab # [ 272.070349] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.801Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"bb8c7283ce8e2bb9c8f8386f","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-bb8c7283ce8e2bb9c8f8386f: done: 0.015027 sec","job_status":"done","queue_duration_s":1.452794,"scheduling_latency_s":1.452811,"redis_calls":13,"redis_duration_s":0.001594,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000588,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001006,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010443,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015027,"completed_at":"2026-10-02T16:57:05.801Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1475gitlab # [ 272.111179] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e0946474a55efd0e116367e","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e0946474a55efd0e116367e: start","job_status":"start","queue_duration_s":1.46901,"scheduling_latency_s":1.469029}1476gitlab # [ 272.124122] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.817Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8e0946474a55efd0e116367e","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8e0946474a55efd0e116367e: done: 0.014962 sec","job_status":"done","queue_duration_s":1.46901,"scheduling_latency_s":1.469029,"redis_calls":13,"redis_duration_s":0.0015760000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000599,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010365,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014962,"completed_at":"2026-10-02T16:57:05.817Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1477gitlab # [ 272.165437] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.818Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"321ded19c2a38cb669b7baf6","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-321ded19c2a38cb669b7baf6: start","job_status":"start","queue_duration_s":1.485166,"scheduling_latency_s":1.485184}1478gitlab # [ 272.178478] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.833Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"321ded19c2a38cb669b7baf6","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-321ded19c2a38cb669b7baf6: done: 0.014978 sec","job_status":"done","queue_duration_s":1.485166,"scheduling_latency_s":1.485184,"redis_calls":13,"redis_duration_s":0.001573,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001009,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010085,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014978,"completed_at":"2026-10-02T16:57:05.833Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1479gitlab # [ 272.219349] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.835Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90902d0104965982017c82df","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90902d0104965982017c82df: start","job_status":"start","queue_duration_s":1.501503,"scheduling_latency_s":1.50152}1480gitlab # [ 272.232465] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.850Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"90902d0104965982017c82df","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-90902d0104965982017c82df: done: 0.015031 sec","job_status":"done","queue_duration_s":1.501503,"scheduling_latency_s":1.50152,"redis_calls":13,"redis_duration_s":0.0017749999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000539,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001236,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010392,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015031,"completed_at":"2026-10-02T16:57:05.850Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1481gitlab # [ 272.274203] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.851Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b312f1315a730b59110d287","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b312f1315a730b59110d287: start","job_status":"start","queue_duration_s":1.517693,"scheduling_latency_s":1.51771}1482gitlab # [ 272.287250] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.865Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3b312f1315a730b59110d287","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3b312f1315a730b59110d287: done: 0.014558 sec","job_status":"done","queue_duration_s":1.517693,"scheduling_latency_s":1.51771,"redis_calls":13,"redis_duration_s":0.0014980000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000537,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014558,"completed_at":"2026-10-02T16:57:05.865Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1483gitlab # [ 272.328396] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.867Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d205bdb50d48fab7f57907ea","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d205bdb50d48fab7f57907ea: start","job_status":"start","queue_duration_s":1.533509,"scheduling_latency_s":1.533527}1484gitlab # [ 272.341259] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.881Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"d205bdb50d48fab7f57907ea","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-d205bdb50d48fab7f57907ea: done: 0.014691 sec","job_status":"done","queue_duration_s":1.533509,"scheduling_latency_s":1.533527,"redis_calls":13,"redis_duration_s":0.0015019999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010261,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014691,"completed_at":"2026-10-02T16:57:05.881Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1485gitlab # [ 272.382085] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.883Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e6a92ef4c6e29486348bd3","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e6a92ef4c6e29486348bd3: start","job_status":"start","queue_duration_s":1.549346,"scheduling_latency_s":1.549363}1486gitlab # [ 272.395122] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.898Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a6e6a92ef4c6e29486348bd3","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a6e6a92ef4c6e29486348bd3: done: 0.014965 sec","job_status":"done","queue_duration_s":1.549346,"scheduling_latency_s":1.549363,"redis_calls":13,"redis_duration_s":0.001533,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000549,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010163,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014965,"completed_at":"2026-10-02T16:57:05.897Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1487gitlab # [ 272.436043] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.899Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85045b45061c62084b92d90c","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85045b45061c62084b92d90c: start","job_status":"start","queue_duration_s":1.56544,"scheduling_latency_s":1.565457}1488gitlab # [ 272.448865] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.913Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"85045b45061c62084b92d90c","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-85045b45061c62084b92d90c: done: 0.014627 sec","job_status":"done","queue_duration_s":1.56544,"scheduling_latency_s":1.565457,"redis_calls":13,"redis_duration_s":0.0015509999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000575,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010176,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014627,"completed_at":"2026-10-02T16:57:05.913Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1489gitlab # [ 272.490792] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.914Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a5d350621dc9b5608de7bc1","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a5d350621dc9b5608de7bc1: start","job_status":"start","queue_duration_s":1.581194,"scheduling_latency_s":1.581211}1490gitlab # [ 272.503918] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.929Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6a5d350621dc9b5608de7bc1","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6a5d350621dc9b5608de7bc1: done: 0.01454 sec","job_status":"done","queue_duration_s":1.581194,"scheduling_latency_s":1.581211,"redis_calls":13,"redis_duration_s":0.001482,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00092,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01454,"completed_at":"2026-10-02T16:57:05.929Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1491gitlab # [ 272.545039] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.930Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe1e5393364d1243479dfccc","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe1e5393364d1243479dfccc: start","job_status":"start","queue_duration_s":1.596902,"scheduling_latency_s":1.59692}1492gitlab # [ 272.557973] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.945Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fe1e5393364d1243479dfccc","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fe1e5393364d1243479dfccc: done: 0.01495 sec","job_status":"done","queue_duration_s":1.596902,"scheduling_latency_s":1.59692,"redis_calls":13,"redis_duration_s":0.00151,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000965,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009907,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01495,"completed_at":"2026-10-02T16:57:05.945Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1493gitlab # [ 272.598821] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.946Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a923a6f08ddc74268f44a19","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a923a6f08ddc74268f44a19: start","job_status":"start","queue_duration_s":1.613244,"scheduling_latency_s":1.613261}1494gitlab # [ 272.611926] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.961Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5a923a6f08ddc74268f44a19","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5a923a6f08ddc74268f44a19: done: 0.014692 sec","job_status":"done","queue_duration_s":1.613244,"scheduling_latency_s":1.613261,"redis_calls":13,"redis_duration_s":0.001535,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000587,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010233,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014692,"completed_at":"2026-10-02T16:57:05.961Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1495gitlab # [ 272.652928] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.962Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97e1ba04d0d25b5d95f35f77","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97e1ba04d0d25b5d95f35f77: start","job_status":"start","queue_duration_s":1.629021,"scheduling_latency_s":1.629037}1496gitlab # [ 272.665920] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.977Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"97e1ba04d0d25b5d95f35f77","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-97e1ba04d0d25b5d95f35f77: done: 0.014525 sec","job_status":"done","queue_duration_s":1.629021,"scheduling_latency_s":1.629037,"redis_calls":13,"redis_duration_s":0.001701,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000547,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001154,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010115,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014525,"completed_at":"2026-10-02T16:57:05.977Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1497gitlab # [ 272.707659] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.978Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5758caa0171022190b5f0691","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5758caa0171022190b5f0691: start","job_status":"start","queue_duration_s":1.64465,"scheduling_latency_s":1.644667}1498gitlab # [ 272.720827] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.992Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"5758caa0171022190b5f0691","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-5758caa0171022190b5f0691: done: 0.014358 sec","job_status":"done","queue_duration_s":1.64465,"scheduling_latency_s":1.644667,"redis_calls":13,"redis_duration_s":0.001487,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000933,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009998,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014358,"completed_at":"2026-10-02T16:57:05.992Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1499gitlab # [ 272.761251] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:05.993Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ac66b855c2cb2d353988806","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ac66b855c2cb2d353988806: start","job_status":"start","queue_duration_s":1.660239,"scheduling_latency_s":1.660256}1500gitlab # [ 272.774202] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.008Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8ac66b855c2cb2d353988806","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8ac66b855c2cb2d353988806: done: 0.014456 sec","job_status":"done","queue_duration_s":1.660239,"scheduling_latency_s":1.660256,"redis_calls":13,"redis_duration_s":0.001497,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000562,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000935,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009836,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014456,"completed_at":"2026-10-02T16:57:06.008Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1501gitlab # [ 272.814958] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.009Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"542f552ce7d0c98c56055c4d","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-542f552ce7d0c98c56055c4d: start","job_status":"start","queue_duration_s":1.676035,"scheduling_latency_s":1.676052}1502gitlab # [ 272.828105] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.024Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"542f552ce7d0c98c56055c4d","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-542f552ce7d0c98c56055c4d: done: 0.014844 sec","job_status":"done","queue_duration_s":1.676035,"scheduling_latency_s":1.676052,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000564,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000991,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010371,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014844,"completed_at":"2026-10-02T16:57:06.024Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1503gitlab # [ 272.868924] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.025Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b904ebd5c5f9dd33ed93f370","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b904ebd5c5f9dd33ed93f370: start","job_status":"start","queue_duration_s":1.692024,"scheduling_latency_s":1.692041}1504gitlab # [ 272.882820] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.040Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b904ebd5c5f9dd33ed93f370","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b904ebd5c5f9dd33ed93f370: done: 0.014668 sec","job_status":"done","queue_duration_s":1.692024,"scheduling_latency_s":1.692041,"redis_calls":13,"redis_duration_s":0.001519,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000976,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010274,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014668,"completed_at":"2026-10-02T16:57:06.040Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1505gitlab # [ 272.923718] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.041Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff549a064fb62594a15d9d39","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff549a064fb62594a15d9d39: start","job_status":"start","queue_duration_s":1.707813,"scheduling_latency_s":1.707831}1506gitlab # [ 272.936622] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.056Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ff549a064fb62594a15d9d39","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ff549a064fb62594a15d9d39: done: 0.015079 sec","job_status":"done","queue_duration_s":1.707813,"scheduling_latency_s":1.707831,"redis_calls":13,"redis_duration_s":0.001511,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000544,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000967,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010514,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015079,"completed_at":"2026-10-02T16:57:06.056Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1507gitlab # [ 272.977442] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.057Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2ef9c53c7a65b7349445cd9","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2ef9c53c7a65b7349445cd9: start","job_status":"start","queue_duration_s":1.72404,"scheduling_latency_s":1.724058}1508gitlab # [ 272.990297] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.072Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2ef9c53c7a65b7349445cd9","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2ef9c53c7a65b7349445cd9: done: 0.015193 sec","job_status":"done","queue_duration_s":1.72404,"scheduling_latency_s":1.724058,"redis_calls":13,"redis_duration_s":0.001556,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000574,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000982,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009973,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015193,"completed_at":"2026-10-02T16:57:06.072Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1509gitlab # [ 273.031029] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.074Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d32a95a4719a8729fbcbab","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d32a95a4719a8729fbcbab: start","job_status":"start","queue_duration_s":1.740388,"scheduling_latency_s":1.740405}1510gitlab # [ 273.044109] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.088Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"63d32a95a4719a8729fbcbab","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-63d32a95a4719a8729fbcbab: done: 0.014587 sec","job_status":"done","queue_duration_s":1.740388,"scheduling_latency_s":1.740405,"redis_calls":13,"redis_duration_s":0.001491,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000543,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000948,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010212,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014587,"completed_at":"2026-10-02T16:57:06.088Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1511gitlab # [ 273.085745] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.089Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c6b79551efc3a2c4b6deb3","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c6b79551efc3a2c4b6deb3: start","job_status":"start","queue_duration_s":1.756107,"scheduling_latency_s":1.756125}1512gitlab # [ 273.098686] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.104Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"72c6b79551efc3a2c4b6deb3","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-72c6b79551efc3a2c4b6deb3: done: 0.01465 sec","job_status":"done","queue_duration_s":1.756107,"scheduling_latency_s":1.756125,"redis_calls":13,"redis_duration_s":0.001532,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000579,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010256,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01465,"completed_at":"2026-10-02T16:57:06.104Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1513gitlab # [ 273.139532] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.105Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b302473c3b44a0b57158b3d8","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b302473c3b44a0b57158b3d8: start","job_status":"start","queue_duration_s":1.771887,"scheduling_latency_s":1.771905}1514gitlab # [ 273.152633] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.120Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b302473c3b44a0b57158b3d8","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b302473c3b44a0b57158b3d8: done: 0.014503 sec","job_status":"done","queue_duration_s":1.771887,"scheduling_latency_s":1.771905,"redis_calls":13,"redis_duration_s":0.001492,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000956,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009889,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014503,"completed_at":"2026-10-02T16:57:06.120Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1515gitlab # [ 273.193356] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.121Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3682bd177bbac8b84f0082a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3682bd177bbac8b84f0082a: start","job_status":"start","queue_duration_s":1.787928,"scheduling_latency_s":1.787946}1516gitlab # [ 273.206307] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.136Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a3682bd177bbac8b84f0082a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a3682bd177bbac8b84f0082a: done: 0.014619 sec","job_status":"done","queue_duration_s":1.787928,"scheduling_latency_s":1.787946,"redis_calls":13,"redis_duration_s":0.0017490000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000568,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001181,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010165,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014619,"completed_at":"2026-10-02T16:57:06.136Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1517gitlab # [ 273.247181] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.137Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4a32348010b5403d7af5b2b","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4a32348010b5403d7af5b2b: start","job_status":"start","queue_duration_s":1.803769,"scheduling_latency_s":1.803786}1518gitlab # [ 273.260284] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.152Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c4a32348010b5403d7af5b2b","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c4a32348010b5403d7af5b2b: done: 0.014969 sec","job_status":"done","queue_duration_s":1.803769,"scheduling_latency_s":1.803786,"redis_calls":13,"redis_duration_s":0.0017729999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000569,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001204,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010337,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014969,"completed_at":"2026-10-02T16:57:06.152Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1519gitlab # [ 273.301725] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b079e0608e2d05be35b9bbc","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b079e0608e2d05be35b9bbc: start","job_status":"start","queue_duration_s":1.819869,"scheduling_latency_s":1.819886}1520gitlab # [ 273.314868] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.168Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8b079e0608e2d05be35b9bbc","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8b079e0608e2d05be35b9bbc: done: 0.014621 sec","job_status":"done","queue_duration_s":1.819869,"scheduling_latency_s":1.819886,"redis_calls":13,"redis_duration_s":0.001486,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00055,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010291,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014621,"completed_at":"2026-10-02T16:57:06.168Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1521gitlab # [ 273.355902] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.169Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fb3d2912a71f877d4257084","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fb3d2912a71f877d4257084: start","job_status":"start","queue_duration_s":1.835655,"scheduling_latency_s":1.835899}1522gitlab # [ 273.369164] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.184Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1fb3d2912a71f877d4257084","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1fb3d2912a71f877d4257084: done: 0.014848 sec","job_status":"done","queue_duration_s":1.835655,"scheduling_latency_s":1.835899,"redis_calls":13,"redis_duration_s":0.001499,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000553,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000946,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010019,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014848,"completed_at":"2026-10-02T16:57:06.184Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1523gitlab # [ 273.410187] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.185Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50bfe2a3a373019aea9e289","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50bfe2a3a373019aea9e289: start","job_status":"start","queue_duration_s":1.851667,"scheduling_latency_s":1.851685}1524gitlab # [ 273.422984] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.200Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a50bfe2a3a373019aea9e289","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a50bfe2a3a373019aea9e289: done: 0.014696 sec","job_status":"done","queue_duration_s":1.851667,"scheduling_latency_s":1.851685,"redis_calls":13,"redis_duration_s":0.001558,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000538,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00102,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01027,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014696,"completed_at":"2026-10-02T16:57:06.200Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1525gitlab # [ 273.464087] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5e6685438425957f91c70bd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5e6685438425957f91c70bd: start","job_status":"start","queue_duration_s":1.867519,"scheduling_latency_s":1.867537}1526gitlab # [ 273.477018] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.215Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c5e6685438425957f91c70bd","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c5e6685438425957f91c70bd: done: 0.014599 sec","job_status":"done","queue_duration_s":1.867519,"scheduling_latency_s":1.867537,"redis_calls":13,"redis_duration_s":0.001555,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000571,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009985,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014599,"completed_at":"2026-10-02T16:57:06.215Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1527gitlab # [ 273.518837] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.217Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee08684be13de1057c51160","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee08684be13de1057c51160: start","job_status":"start","queue_duration_s":1.883504,"scheduling_latency_s":1.883521}1528gitlab # [ 273.531932] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.231Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6ee08684be13de1057c51160","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6ee08684be13de1057c51160: done: 0.0144 sec","job_status":"done","queue_duration_s":1.883504,"scheduling_latency_s":1.883521,"redis_calls":13,"redis_duration_s":0.0015049999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000552,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000953,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01008,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.0144,"completed_at":"2026-10-02T16:57:06.231Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1529gitlab # [ 273.573084] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.232Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83617ad5445db4e256c19a50","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83617ad5445db4e256c19a50: start","job_status":"start","queue_duration_s":1.899023,"scheduling_latency_s":1.89904}1530gitlab # [ 273.586083] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.247Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"83617ad5445db4e256c19a50","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-83617ad5445db4e256c19a50: done: 0.014681 sec","job_status":"done","queue_duration_s":1.899023,"scheduling_latency_s":1.89904,"redis_calls":13,"redis_duration_s":0.001503,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000558,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000945,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014681,"completed_at":"2026-10-02T16:57:06.247Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1531gitlab # [ 273.626870] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.248Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04d83f4d9de0c8a88b5b302","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04d83f4d9de0c8a88b5b302: start","job_status":"start","queue_duration_s":1.914852,"scheduling_latency_s":1.914869}1532gitlab # [ 273.639891] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.263Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a04d83f4d9de0c8a88b5b302","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a04d83f4d9de0c8a88b5b302: done: 0.01452 sec","job_status":"done","queue_duration_s":1.914852,"scheduling_latency_s":1.914869,"redis_calls":13,"redis_duration_s":0.001539,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000582,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000957,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01012,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01452,"completed_at":"2026-10-02T16:57:06.263Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1533gitlab # [ 273.680932] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.264Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8df78b20f22b7ff24d0aff7a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8df78b20f22b7ff24d0aff7a: start","job_status":"start","queue_duration_s":1.930562,"scheduling_latency_s":1.93058}1534gitlab # [ 273.694773] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.282Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8df78b20f22b7ff24d0aff7a","created_at":"2026-10-02T16:57:04.052Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-1f8c6e40ef6f4555","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"1f6958a1625350a5a1accb4c83a9328f","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:04.333Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8df78b20f22b7ff24d0aff7a: done: 0.018477 sec","job_status":"done","queue_duration_s":1.930562,"scheduling_latency_s":1.93058,"redis_calls":13,"redis_duration_s":0.0016719999999999999,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000556,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001116,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010515,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018477,"completed_at":"2026-10-02T16:57:06.282Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1535gitlab # [ 273.735879] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.283Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"46c2da10399a9b40d4a5ff4e","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-46c2da10399a9b40d4a5ff4e: start","job_status":"start","queue_duration_s":1.011181,"scheduling_latency_s":1.011199}1536gitlab # [ 273.748856] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.295Z","message":"No existing merge request to be cleaned up."}1537gitlab # [ 273.750543] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.303Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"46c2da10399a9b40d4a5ff4e","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-46c2da10399a9b40d4a5ff4e: done: 0.02016 sec","job_status":"done","queue_duration_s":1.011181,"scheduling_latency_s":1.011199,"redis_calls":10,"redis_duration_s":0.00145,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000635,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000815,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014402,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.02016,"completed_at":"2026-10-02T16:57:06.303Z","load_balancing_strategy":"primary","db_duration_s":0.002139,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1538gitlab # [ 273.790801] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.305Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b27431aa31356cf5e81faf1f","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b27431aa31356cf5e81faf1f: start","job_status":"start","queue_duration_s":1.032543,"scheduling_latency_s":1.032561}1539gitlab # [ 273.803784] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.316Z","message":"No existing merge request to be cleaned up."}1540gitlab # [ 273.805580] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.324Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b27431aa31356cf5e81faf1f","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b27431aa31356cf5e81faf1f: done: 0.019511 sec","job_status":"done","queue_duration_s":1.032543,"scheduling_latency_s":1.032561,"redis_calls":10,"redis_duration_s":0.001415,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000814,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014009,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019511,"completed_at":"2026-10-02T16:57:06.324Z","load_balancing_strategy":"primary","db_duration_s":0.001514,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1541gitlab # [ 273.845612] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.326Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b02f07fac2b8eaa13d21b882","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b02f07fac2b8eaa13d21b882: start","job_status":"start","queue_duration_s":1.053664,"scheduling_latency_s":1.053682}1542gitlab # [ 273.858343] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.337Z","message":"No existing merge request to be cleaned up."}1543gitlab # [ 273.860106] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.346Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"b02f07fac2b8eaa13d21b882","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-b02f07fac2b8eaa13d21b882: done: 0.020678 sec","job_status":"done","queue_duration_s":1.053664,"scheduling_latency_s":1.053682,"redis_calls":10,"redis_duration_s":0.0019820000000000003,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000683,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001299,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015072,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020678,"completed_at":"2026-10-02T16:57:06.346Z","load_balancing_strategy":"primary","db_duration_s":0.001516,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1544gitlab # [ 273.901487] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.348Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4562f7c2cc54c7c522432507","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4562f7c2cc54c7c522432507: start","job_status":"start","queue_duration_s":1.075992,"scheduling_latency_s":1.07601}1545gitlab # [ 273.914155] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.360Z","message":"No existing merge request to be cleaned up."}1546gitlab # [ 273.915855] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:06.369Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"4562f7c2cc54c7c522432507","created_at":"2026-10-02T16:57:05.261Z","trace_propagation_headers":{"sentry-trace":"a36769dfc7eb4f5a9d43978101534e12-042e2bfb9fbc4bb0","baggage":"sentry-trace_id=a36769dfc7eb4f5a9d43978101534e12,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"5e4c55b4635e0ea2b08e0f71c7997236","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:05.272Z","job_size_bytes":2,"pid":1909,"sidekiq_tid":"mal9","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-4562f7c2cc54c7c522432507: done: 0.020889 sec","job_status":"done","queue_duration_s":1.075992,"scheduling_latency_s":1.07601,"redis_calls":10,"redis_duration_s":0.001549,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000602,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000947,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014629,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.020889,"completed_at":"2026-10-02T16:57:06.369Z","load_balancing_strategy":"primary","db_duration_s":0.001504,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1547gitlab # [ 299.531779] postgres[3246]: [3246] ERROR: relation "table_without_models" does not exist1548gitlab # [ 299.532222] postgres[3246]: [3246] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1549gitlab # [ 305.168308] runuser[2893]: pam_unix(runuser:session): session closed for user gitlab1550gitlab # [ 305.169463] setup-gitlab-start[2878]: + systemctl restart gitlab-sidekiq.service1551gitlab # [ 305.195273] systemd[1]: Stopping gitlab-sidekiq.service...1552gitlab # [ 305.197285] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:40.131Z","message":"Shutting down"}1553gitlab # [ 305.199583] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:40.132Z","message":"Scheduler exiting..."}1554gitlab # [ 305.202009] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:40.132Z","message":"Terminating quiet threads for default capsule"}1555gitlab # [ 305.204718] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:40.133Z","message":"Scheduler exiting..."}1556gitlab # [ 305.697530] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:40.633Z","message":"Pausing to allow jobs to finish..."}1557gitlab # [ 306.702930] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:41.636Z","message":"Bye!"}1558gitlab # [ 306.726803] sidekiq-cluster[1909]: {"severity":"INFO","time":"2026-10-02T16:57:41.663Z","message":"stopped","memwd_reason":"background task stopped","memwd_handler_class":"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler","memwd_sleep_time_s":3,"pid":1909,"worker_id":"sidekiq_0","memwd_rss_bytes":920424448,"retry":0}1559gitlab # [ 307.496385] systemd[1]: gitlab-sidekiq.service: Deactivated successfully.1560gitlab # [ 307.496702] systemd[1]: Stopped gitlab-sidekiq.service.1561gitlab # [ 307.496828] systemd[1]: gitlab-sidekiq.service: Consumed 1min 49.007s CPU time over 3min 21.529s wall clock time, 937.9M memory peak, 4.7M read from disk.1562gitlab # [ 307.504221] systemd[1]: Started gitlab-sidekiq.service.1563gitlab # [ 307.507017] setup-gitlab-start[2878]: + mkdir -p /var/lib/secrets1564gitlab # [ 307.510810] setup-gitlab-start[2878]: + echo nixbot-test-token-of-20-chars1565gitlab # [ 307.512627] setup-gitlab-start[2878]: + chmod 644 /var/lib/secrets/gitlab-token1566gitlab # [ 307.514202] setup-gitlab-start[2878]: + timeout 300 bash -c $'until curl -fs -H "PRIVATE-TOKEN: $0" http://gitlab/api/v4/projects/root%2Ftest-flake > /dev/null; do\n curl -s -X POST -H "PRIVATE-TOKEN: $0" -H "Content-Type: application/json" \\\n -d "{\\"name\\":\\"test-flake\\",\\"visibility\\":\\"public\\",\\"topics\\":[\\"build-with-buildbot\\"]}" \\\n http://gitlab/api/v4/projects | jq -c "{id, message, error}"\n sleep 5\ndone' nixbot-test-token-of-20-chars1567gitlab # [ 307.896041] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:42 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 404 35 "" "curl/8.21.0" 3561568gitlab # [ 308.692428] gitaly[916]: time="2026-10-02T16:57:43.628Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.628" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1569gitlab # [ 308.692738] gitaly[916]: time="2026-10-02T16:57:43.628Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.007" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=95 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.628" grpc.time_ms=0.391 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1570gitlab # [ 308.828251] gitaly[916]: time="2026-10-02T16:57:43.764Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.764" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1571gitlab # [ 308.839055] gitaly[916]: time="2026-10-02T16:57:43.765Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.144" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.764" grpc.time_ms=0.418 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1572gitlab # [ 308.873105] gitaly[916]: time="2026-10-02T16:57:43.809Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K diskcache=faf68fe4-07d3-42d6-8e6b-bc9afd8124e3 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:57:53.153" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.774" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1573gitlab # [ 308.890168] gitaly[916]: time="2026-10-02T16:57:43.810Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1438 command.oublock=32 command.real_time_ms=22 command.spawn_token_fork_ms=1 command.system_time_ms=16 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:57:53.153" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.774" grpc.time_ms=35.776 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1574gitlab # [ 308.911828] gitaly[916]: time="2026-10-02T16:57:43.822Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.202" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.822" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1575gitlab # [ 308.927595] gitaly[916]: time="2026-10-02T16:57:43.823Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.202" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.822" grpc.time_ms=0.584 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1576gitlab # [ 308.942430] gitaly[916]: time="2026-10-02T16:57:43.837Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=2 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:57:53.207" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.828" grpc.time_ms=8.997 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1577gitlab # [ 308.993519] gitaly[916]: time="2026-10-02T16:57:43.930Z" level=error msg="housekeeping: find repo path" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K error="repository not found" grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.929" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1578gitlab # [ 309.010317] gitaly[916]: time="2026-10-02T16:57:43.930Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.009" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.929" grpc.time_ms=0.419 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1579gitlab # [ 309.034732] gitaly[916]: time="2026-10-02T16:57:43.971Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K diskcache=2d5bfa3c-595f-4826-b0a6-ace46a5b2c69 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:57:53.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.932" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1580gitlab # [ 309.051797] gitaly[916]: time="2026-10-02T16:57:43.971Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=18 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1435 command.oublock=32 command.real_time_ms=24 command.spawn_token_fork_ms=0 command.system_time_ms=13 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CreateRepository grpc.request.deadline="2026-10-02T16:57:53.012" grpc.request.fullMethod=/gitaly.RepositoryService/CreateRepository grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=137 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.932" grpc.time_ms=38.581 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1581gitlab # [ 309.071438] gitaly[916]: time="2026-10-02T16:57:43.979Z" level=info msg="collected repository statistics" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.058" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.978" pid=916 remote_ip="2001:db8:1::1" repository_stats="map[directory_count:7 file_count:3]" span.kind=server system=grpc username=root1582gitlab # [ 309.084367] gitaly[916]: time="2026-10-02T16:57:43.979Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:53.058" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.978" grpc.time_ms=0.819 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1583gitlab # [ 309.096846] gitaly[916]: time="2026-10-02T16:57:43.994Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:57:53.064" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake.wiki grpc.request.glRepository=wiki-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.wiki.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:43.984" grpc.time_ms=10.359 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1584gitlab # [ 309.112445] gitaly[916]: time="2026-10-02T16:57:44.036Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:57:54.005" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:44.036" grpc.time_ms=0.259 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1585gitlab # [ 309.124532] gitaly[916]: time="2026-10-02T16:57:44.046Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:57:54.015" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:44.045" grpc.time_ms=0.342 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1586gitlab # [ 309.136579] redis[786]: 10 changes in 300 seconds. Saving...1587gitlab # [ 309.137648] redis[786]: Background saving started by pid 33381588gitlab # [ 309.138922] redis[3338]: BGSAVE done, 372 keys saved, 0 keys skipped, 99873 bytes written.1589gitlab # [ 309.143328] redis[3338]: DB saved on disk1590gitlab # [ 309.144125] redis[3338]: Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB1591gitlab # [ 309.232477] redis[786]: Background saving terminated with success1592gitlab # [ 309.712986] gitaly[916]: time="2026-10-02T16:57:44.648Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSQP3AWH4A2297K8FQB5K grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:57:54.018" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:44.638" grpc.time_ms=10.096 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1593gitlab # [ 310.294841] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:45 +0000] "POST /api/v4/projects HTTP/1.1" 201 4828 "" "curl/8.21.0" 23791594gitlab # [ 310.298244] setup-gitlab-start[3309]: {"id":1,"message":null,"error":null}1595gitlab # [ 310.526140] postgres[864]: [864] LOG: checkpoint starting: time1596gitlab # [ 311.562571] gitaly[916]: time="2026-10-02T16:57:46.498Z" level=info msg="burden monitor stats" pid=9161597gitlab # [ 312.750639] sidekiq-cluster[3297]: {"severity":"INFO","time":"2026-10-02T16:57:47.657Z","message":"Starting cluster with 1 processes"}1598gitlab # [ 315.447634] gitaly[916]: time="2026-10-02T16:57:50.383Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSYXH0Q328GY6KTMJR6DD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:58:00.002" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:50.372" grpc.time_ms=10.695 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1599gitlab # [ 315.524194] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:50 +0000] "GET /api/v4/projects/root%2Ftest-flake HTTP/1.1" 200 4924 "" "curl/8.21.0" 2021600gitlab # [ 315.528536] setup-gitlab-start[2878]: + mkdir -p /tmp/test-flake1601gitlab # [ 315.531266] setup-gitlab-start[2878]: + cd /tmp/test-flake1602gitlab # [ 315.532700] setup-gitlab-start[2878]: + git init -b master1603gitlab # [ 315.547914] setup-gitlab-start[3383]: Initialized empty Git repository in /tmp/test-flake/.git/1604gitlab # [ 315.549946] setup-gitlab-start[2878]: + git config user.name test1605gitlab # [ 315.551979] setup-gitlab-start[2878]: + git config user.email test@example.com1606gitlab # [ 315.555860] setup-gitlab-start[2878]: + cat1607gitlab # [ 315.558920] setup-gitlab-start[2878]: + git add flake.nix1608gitlab # [ 315.563428] setup-gitlab-start[2878]: + git commit -m 'initial commit'1609gitlab # [ 315.580520] setup-gitlab-start[3388]: [master (root-commit) 4fe91ef] initial commit1610gitlab # [ 315.582195] setup-gitlab-start[3388]: 1 file changed, 10 insertions(+)1611gitlab # [ 315.583661] setup-gitlab-start[3388]: create mode 100644 flake.nix1612gitlab # [ 315.586142] setup-gitlab-start[2878]: + git remote add origin http://oauth2:nixbot-test-token-of-20-chars@gitlab/root/test-flake.git1613gitlab # [ 315.592368] setup-gitlab-start[2878]: + git push -u origin master1614gitlab # [ 315.819663] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:50 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 1941615gitlab # [ 315.970926] gitaly[916]: time="2026-10-02T16:57:50.907Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZD55Z5J26SZKMTXXEJF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=242 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:50.895" grpc.time_ms=11.608 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1616gitlab # [ 315.993115] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:50 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 178 "" "git/2.55.0" 1501617gitlab # [ 316.216946] gitaly[916]: time="2026-10-02T16:57:51.153Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:58:01.112" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=169 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:51.143" grpc.time_ms=10.161 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1618gitlab # [ 316.291755] gitaly[916]: time="2026-10-02T16:57:51.225Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=300 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:58:21.014" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:57:51.215" grpc.time_ms=10.616 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1619gitlab # [ 316.318027] gitaly[916]: time="2026-10-02T16:57:51.254Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q duration_ms=193ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed"1620gitlab # [ 316.360615] gitaly[916]: time="2026-10-02T16:57:51.297Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q duration_ms=41ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive"1621gitlab # [ 316.365401] gitaly[916]: time="2026-10-02T16:57:51.297Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:51.059" grpc.time_ms=237.659 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1622gitlab # [ 316.398655] gitaly[916]: time="2026-10-02T16:57:51.335Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:51.334" grpc.time_ms=0.906 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1623gitlab # [ 316.451390] gitaly[916]: time="2026-10-02T16:57:51.387Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:51.386" grpc.time_ms=0.838 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1624gitlab # [ 316.480050] gitaly[916]: time="2026-10-02T16:57:51.416Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:51.415" grpc.time_ms=0.607 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1625gitlab # [ 316.741987] gitaly[916]: time="2026-10-02T16:57:51.678Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:58:01.007" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:51.668" grpc.time_ms=9.947 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1626gitlab # [ 316.796465] gitaly[916]: time="2026-10-02T16:57:51.733Z" level=info msg="finished unary call with code OK" command.count=5 command.cpu_time_ms=15 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1431 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:58:01.049" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:51.710" grpc.time_ms=22.676 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1627gitlab # [ 316.850238] gitaly[916]: time="2026-10-02T16:57:51.786Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q duration_ms=321ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive"1628gitlab # [ 316.856247] gitaly[916]: time="2026-10-02T16:57:51.792Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:51.464" grpc.time_ms=328.145 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1629gitlab # [ 316.873430] gitaly[916]: time="2026-10-02T16:57:51.800Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q diskcache=a44d81c9-e90b-4cfd-9145-52363c468303 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:50.995" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1630gitlab # [ 316.890170] setup-gitlab-start[3392]: To http://gitlab/root/test-flake.git1631gitlab # [ 316.891577] setup-gitlab-start[3392]: * [new branch] master -> master1632gitlab # [ 316.893053] setup-gitlab-start[3392]: branch 'master' set up to track 'origin/master'.1633gitlab # [ 316.895268] gitlab-workhorse[822]: time="2026-10-02T16:57:51Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521634gitlab # [ 316.898293] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:51 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 8811635gitlab # [ 316.901758] systemd[1]: Finished setup-gitlab.service.1636gitlab # [ 316.903060] gitaly[916]: time="2026-10-02T16:57:51.801Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=214 command.inblock=8 command.majflt=0 command.maxrssanon_bytes=847872 command.minflt=13598 command.oublock=32 command.real_time_ms=800 command.spawn_token_fork_ms=0 command.system_time_ms=116 command.user_time_ms=98 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=684 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:50.995" grpc.time_ms=805.613 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1637gitlab # [ 316.923666] systemd[1]: Starting nixbot CI service...1638gitlab: (finished: waiting for unit setup-gitlab.service, in 318.29 seconds)1639gitlab: waiting for unit nixbot.service1640gitlab # [ 317.965680] nixbot[3482]: {"ts": "2026-10-02T16:57:52.901Z", "level": "info", "logger": "nixbot.main", "msg": "nixbot starting", "url": "http://localhost:8010", "state_dir": "/var/lib/nixbot"}1641gitlab # [ 317.999478] nixbot[3482]: {"ts": "2026-10-02T16:57:52.935Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 1, "migration": "initial"}1642gitlab # [ 318.023164] nixbot[3482]: {"ts": "2026-10-02T16:57:52.959Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 2, "migration": "scheduled_effects"}1643gitlab # [ 318.028429] nixbot[3482]: {"ts": "2026-10-02T16:57:52.964Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 3, "migration": "gitea_webhook_secrets"}1644gitlab # [ 318.032959] nixbot[3482]: {"ts": "2026-10-02T16:57:52.969Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 4, "migration": "api_tokens"}1645gitlab # [ 318.039487] nixbot[3482]: {"ts": "2026-10-02T16:57:52.976Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 5, "migration": "status_notify"}1646gitlab # [ 318.043849] nixbot[3482]: {"ts": "2026-10-02T16:57:52.980Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 6, "migration": "webhook_secrets"}1647gitlab # [ 318.047778] nixbot[3482]: {"ts": "2026-10-02T16:57:52.982Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 7, "migration": "build_effects"}1648gitlab # [ 318.051816] nixbot[3482]: {"ts": "2026-10-02T16:57:52.988Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 8, "migration": "api_token_groups"}1649gitlab # [ 318.055530] nixbot[3482]: {"ts": "2026-10-02T16:57:52.990Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 9, "migration": "scheduled_effect_runs"}1650gitlab # [ 318.059194] nixbot[3482]: {"ts": "2026-10-02T16:57:52.994Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 10, "migration": "work_queue"}1651gitlab # [ 318.063690] nixbot[3482]: {"ts": "2026-10-02T16:57:53.000Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 11, "migration": "revoked_sessions"}1652gitlab # [ 318.067228] nixbot[3482]: {"ts": "2026-10-02T16:57:53.003Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 12, "migration": "truncate_notify_names"}1653gitlab # [ 318.070642] nixbot[3482]: {"ts": "2026-10-02T16:57:53.005Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 13, "migration": "failed_builds_per_project"}1654gitlab # [ 318.074276] nixbot[3482]: {"ts": "2026-10-02T16:57:53.010Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 14, "migration": "streamed_eval_warnings"}1655gitlab # [ 318.078011] nixbot[3482]: {"ts": "2026-10-02T16:57:53.011Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 15, "migration": "eval_completed"}1656gitlab # [ 318.081535] nixbot[3482]: {"ts": "2026-10-02T16:57:53.013Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 16, "migration": "reconcile_watermark"}1657gitlab # [ 318.085082] nixbot[3482]: {"ts": "2026-10-02T16:57:53.014Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 17, "migration": "check_runs"}1658gitlab # [ 318.088518] nixbot[3482]: {"ts": "2026-10-02T16:57:53.018Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 18, "migration": "effects_commit"}1659gitlab # [ 318.091948] nixbot[3482]: {"ts": "2026-10-02T16:57:53.019Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 19, "migration": "scheduled_run_notify"}1660gitlab # [ 318.095750] nixbot[3482]: {"ts": "2026-10-02T16:57:53.021Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 20, "migration": "log_metadata"}1661gitlab # [ 318.099263] nixbot[3482]: {"ts": "2026-10-02T16:57:53.025Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 21, "migration": "forge_refresh_tokens"}1662gitlab # [ 318.102636] nixbot[3482]: {"ts": "2026-10-02T16:57:53.026Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 22, "migration": "attr_finished_idx"}1663gitlab # [ 318.105853] nixbot[3482]: {"ts": "2026-10-02T16:57:53.028Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 23, "migration": "effect_deps"}1664gitlab # [ 318.108899] nixbot[3482]: {"ts": "2026-10-02T16:57:53.029Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 24, "migration": "drop_forge_tokens"}1665gitlab # [ 318.112098] nixbot[3482]: {"ts": "2026-10-02T16:57:53.031Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 25, "migration": "effect_skipped_rename"}1666gitlab # [ 318.115175] nixbot[3482]: {"ts": "2026-10-02T16:57:53.032Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 26, "migration": "attr_eval_warnings"}1667gitlab # [ 318.118295] nixbot[3482]: {"ts": "2026-10-02T16:57:53.034Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 27, "migration": "eval_stats"}1668gitlab # [ 318.121296] nixbot[3482]: {"ts": "2026-10-02T16:57:53.035Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 28, "migration": "effect_runs"}1669gitlab # [ 318.124243] nixbot[3482]: {"ts": "2026-10-02T16:57:53.050Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 29, "migration": "event_effects"}1670gitlab # [ 318.127140] nixbot[3482]: {"ts": "2026-10-02T16:57:53.055Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 30, "migration": "work_retry"}1671gitlab # [ 318.130067] nixbot[3482]: {"ts": "2026-10-02T16:57:53.057Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 31, "migration": "merged_pr_number"}1672gitlab # [ 318.133153] nixbot[3482]: {"ts": "2026-10-02T16:57:53.058Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 32, "migration": "upload_queue"}1673gitlab # [ 318.135974] nixbot[3482]: {"ts": "2026-10-02T16:57:53.062Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 33, "migration": "effect_runs_catchup"}1674gitlab # [ 318.138931] nixbot[3482]: {"ts": "2026-10-02T16:57:53.064Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 34, "migration": "public_badges"}1675gitlab # [ 318.142014] nixbot[3482]: {"ts": "2026-10-02T16:57:53.070Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 35, "migration": "settle_reset_effects"}1676gitlab # [ 318.144952] nixbot[3482]: {"ts": "2026-10-02T16:57:53.073Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 36, "migration": "pr_approvals"}1677gitlab # [ 318.147574] nixbot[3482]: {"ts": "2026-10-02T16:57:53.077Z", "level": "info", "logger": "nixbot.migrations", "msg": "applying migration", "version": 37, "migration": "effect_skip_key"}1678gitlab # [ 318.486655] nixbot[3482]: INFO: Started server process [3482]1679gitlab # [ 318.487775] nixbot[3482]: INFO: Uvicorn running on socket ('::', 8010, 0, 0) (Press CTRL+C to quit)1680gitlab # [ 318.521592] nixbot[3482]: INFO: ::ffff:127.0.0.1:39568 - "GET /health HTTP/1.1" 200 OK1681gitlab # [ 318.533254] systemd[1]: Started nixbot CI service.1682gitlab # [ 318.534533] systemd[1]: Reached target Multi-User System.1683gitlab # [ 318.535843] systemd[1]: Startup finished in 665ms (kernel) + 4.677s (initrd) + 5min 13.193s (userspace) = 5min 18.535s.1684gitlab # [ 318.832120] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:53 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 3111685gitlab # [ 318.834637] nixbot[3482]: {"ts": "2026-10-02T16:57:53.770Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1686gitlab # [ 318.858642] nixbot[3482]: {"ts": "2026-10-02T16:57:53.795Z", "level": "info", "logger": "nixbot.repos", "msg": "legacy topic import complete", "topics": {"gitlab": "build-with-buildbot"}, "enabled": 1}1687gitlab: (finished: waiting for unit nixbot.service, in 1.06 seconds)1688gitlab: waiting for success: curl --fail -s http://127.0.0.1:8010/health1689gitlab # [ 318.926039] nixbot[3482]: INFO: ::ffff:127.0.0.1:39572 - "GET /health HTTP/1.1" 200 OK1690gitlab: (finished: waiting for success: curl --fail -s http://127.0.0.1:8010/health, in 0.03 seconds)1691subtest: project discovered and webhook registered1692gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks1693gitlab # [ 318.958762] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:53 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 871694gitlab # [ 318.961492] nixbot[3482]: {"ts": "2026-10-02T16:57:53.892Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1695gitlab # [ 318.964583] nixbot[3482]: {"ts": "2026-10-02T16:57:53.893Z", "level": "info", "logger": "nixbot.hook_registration", "msg": "registering webhook", "repo": "root/test-flake"}1696gitlab # [ 319.104195] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "POST /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 201 918 "" "python-httpx/0.28.1" 1391697gitlab # [ 319.106919] nixbot[3482]: {"ts": "2026-10-02T16:57:54.039Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 201 Created\""}1698gitlab # [ 319.192916] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "curl/8.21.0" 2391699gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/hooks, in 0.27 seconds)1700(finished: subtest: project discovered and webhook registered, in 0.27 seconds)1701subtest: push delivers the webhook1702gitlab: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails'1703gitlab # [ 319.285582] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /api/v4/projects?membership=true&archived=false&per_page=100 HTTP/1.1" 200 4909 "" "python-httpx/0.28.1" 1671704gitlab # [ 319.289593] nixbot[3482]: {"ts": "2026-10-02T16:57:54.223Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects?membership=true&archived=false&per_page=100 \"HTTP/1.1 200 OK\""}1705gitlab # [ 319.348340] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/hooks HTTP/1.1" 200 881 "" "python-httpx/0.28.1" 541706gitlab # [ 319.351800] nixbot[3482]: {"ts": "2026-10-02T16:57:54.285Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/hooks \"HTTP/1.1 200 OK\""}1707gitlab # [ 319.444710] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "PUT /api/v4/projects/root%2Ftest-flake/hooks/1 HTTP/1.1" 200 918 "" "python-httpx/0.28.1" 921708gitlab # [ 319.448332] nixbot[3482]: {"ts": "2026-10-02T16:57:54.382Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: PUT http://gitlab/api/v4/projects/root%2Ftest-flake/hooks/1 \"HTTP/1.1 200 OK\""}1709gitlab # [ 319.520274] gitaly[916]: time="2026-10-02T16:57:54.456Z" level=info msg="finished unary call with code OK" catfile.duration_ms=3 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=3 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT2YKA2VRQ9DTHKX85EE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindBranch grpc.request.deadline="2026-10-02T16:58:24.040" grpc.request.fullMethod=/gitaly.RefService/FindBranch grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=215 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:54.440" grpc.time_ms=14.977 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1710gitlab # [ 319.585753] gitaly[916]: time="2026-10-02T16:57:54.522Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT2YKA2VRQ9DTHKX85EE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:24.120" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:57:54.521" grpc.time_ms=0.774 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1711gitlab # [ 319.606601] gitaly[916]: time="2026-10-02T16:57:54.536Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=582 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT2YKA2VRQ9DTHKX85EE5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitIsAncestor grpc.request.deadline="2026-10-02T16:58:04.005" grpc.request.fullMethod=/gitaly.CommitService/CommitIsAncestor grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:57:54.525" grpc.time_ms=10.216 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1712gitlab # [ 319.686552] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/branches/master HTTP/1.1" 200 732 "" "python-httpx/0.28.1" 2351713gitlab # [ 319.690467] nixbot[3482]: {"ts": "2026-10-02T16:57:54.624Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/repository/branches/master \"HTTP/1.1 200 OK\""}1714gitlab # [ 319.794241] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 HTTP/1.1" 200 2 "" "python-httpx/0.28.1" 1001715gitlab # [ 319.798427] nixbot[3482]: {"ts": "2026-10-02T16:57:54.732Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: GET http://gitlab/api/v4/projects/root%2Ftest-flake/merge_requests?state=opened&per_page=100 \"HTTP/1.1 200 OK\""}1716gitlab # [ 319.802745] nixbot[3482]: {"ts": "2026-10-02T16:57:54.736Z", "level": "info", "logger": "nixbot.reconcile", "msg": "reconciliation: building unbuilt head", "project": "root/test-flake", "branch": "master", "pr": null}1717gitlab # [ 319.903211] gitaly[916]: time="2026-10-02T16:57:54.839Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT3APQSWJG8Y6D1V1B2JJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:54.828" grpc.time_ms=11.626 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1718gitlab # [ 319.925910] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 651719gitlab # [ 320.002495] gitaly[916]: time="2026-10-02T16:57:54.939Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3CSANTF71PWESA3QQ81 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:54.927" pid=916 remote_ip="2001:db8:1::1" request_sha=cbf4cb85c7b4a4aa26c72509fcce4a61c2433abb response_bytes=113 span.kind=server system=grpc1720gitlab # [ 320.020703] gitaly[916]: time="2026-10-02T16:57:54.939Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3CSANTF71PWESA3QQ81 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:54.927" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1721gitlab # [ 320.042435] gitlab-workhorse[822]: time="2026-10-02T16:57:54Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=1131722gitlab # [ 320.045824] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:54 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 113 "" "git/2.55.0" 981723gitlab # [ 320.049048] gitaly[916]: time="2026-10-02T16:57:54.939Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3CSANTF71PWESA3QQ81 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:54.927" grpc.time_ms=11.885 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1724gitlab # [ 320.099281] gitaly[916]: time="2026-10-02T16:57:55.035Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3G0C36B7WNV3FNTQMKZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.015" pid=916 remote_ip="2001:db8:1::1" request_sha=b17f8fafb4e3bd46511200b67bd4fea1b27fafa8 response_bytes=223 span.kind=server system=grpc1725gitlab # [ 320.117146] gitaly[916]: time="2026-10-02T16:57:55.035Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3G0C36B7WNV3FNTQMKZ grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.015" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1726gitlab # [ 320.137215] gitlab-workhorse[822]: time="2026-10-02T16:57:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2231727gitlab # [ 320.139823] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 223 "" "git/2.55.0" 1081728gitlab # [ 320.142613] gitaly[916]: time="2026-10-02T16:57:55.052Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1135 command.oublock=0 command.real_time_ms=17 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=6 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3G0C36B7WNV3FNTQMKZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.015" grpc.time_ms=37.552 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1729gitlab # [ 320.233716] gitaly[916]: time="2026-10-02T16:57:55.170Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT3MR7RQQKDWKRQS4S79Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=187 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.158" grpc.time_ms=12.146 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1730gitlab # [ 320.255242] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 771731gitlab # [ 320.321055] gitaly[916]: time="2026-10-02T16:57:55.257Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3Q8XTWN9GCBJ6FQZ078 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.247" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1732gitlab # [ 320.331904] gitaly[916]: time="2026-10-02T16:57:55.257Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3Q8XTWN9GCBJ6FQZ078 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.247" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1733gitlab # [ 320.346050] gitlab-workhorse[822]: time="2026-10-02T16:57:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671734gitlab # [ 320.347980] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 821735gitlab # [ 320.350102] gitaly[916]: time="2026-10-02T16:57:55.258Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=4 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3Q8XTWN9GCBJ6FQZ078 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.247" grpc.time_ms=10.794 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1736gitlab # [ 320.427813] gitaly[916]: time="2026-10-02T16:57:55.364Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=581 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT3V9S4FDR2C82WYKGCDG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=186 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.353" grpc.time_ms=10.579 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1737gitlab # [ 320.441723] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:55 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 591738gitlab # [ 320.521207] gitaly[916]: time="2026-10-02T16:57:55.456Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3X70SA2R0C5ERR553NG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.436" pid=916 remote_ip="2001:db8:1::1" request_sha=fb62c37b60b550dd367e34acce88d0d3e3444d95 response_bytes=209 span.kind=server system=grpc1739gitlab # [ 320.538398] gitaly[916]: time="2026-10-02T16:57:55.457Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3X70SA2R0C5ERR553NG grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.436" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1740gitlab # [ 320.560550] gitlab-workhorse[822]: time="2026-10-02T16:57:55Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2091741gitlab # [ 320.563389] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:55 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 209 "" "git/2.55.0" 901742gitlab # [ 320.566295] gitaly[916]: time="2026-10-02T16:57:55.457Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=17 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1101 command.oublock=0 command.real_time_ms=18 command.spawn_token_fork_ms=0 command.system_time_ms=17 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT3X70SA2R0C5ERR553NG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:57:55.436" grpc.time_ms=21.065 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1743gitlab # [ 320.640316] gitaly[916]: time="2026-10-02T16:57:55.576Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:25.175" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:57:55.575" grpc.time_ms=0.815 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1744gitlab # [ 320.719860] gitaly[916]: time="2026-10-02T16:57:55.656Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:58:25.244" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:55.645" grpc.time_ms=11.222 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1745gitlab # [ 320.749056] gitaly[916]: time="2026-10-02T16:57:55.685Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:58:05.014" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:55.674" grpc.time_ms=10.398 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1746gitlab # [ 320.771033] gitaly[916]: time="2026-10-02T16:57:55.706Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:58:05.036" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:55.696" grpc.time_ms=10.022 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1747gitlab # [ 320.879205] gitaly[916]: time="2026-10-02T16:57:55.815Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:58:05.154" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:55.815" grpc.time_ms=0.483 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1748gitlab # [ 320.899696] gitaly[916]: time="2026-10-02T16:57:55.836Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=570 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:58:05.164" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:57:55.825" grpc.time_ms=10.318 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1749gitlab # [ 320.945649] gitaly[916]: time="2026-10-02T16:57:55.882Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:55.881" grpc.time_ms=0.761 pid=916 span.kind=server system=grpc1750gitlab # [ 320.981677] gitaly[916]: time="2026-10-02T16:57:55.918Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:57:55.917" grpc.time_ms=0.51 pid=916 span.kind=server system=grpc1751gitlab # [ 320.998245] gitaly[916]: time="2026-10-02T16:57:55.927Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 diskcache=1750fbd1-32fe-4046-a5c7-aca04ed54464 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:58:05.179" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:55.839" pid=916 ref=refs/keep-around/4fe91ef492212e25a90e9ab4b648d4a635f4e850 remote_ip="2001:db8:1::1" revision=4fe91ef492212e25a90e9ab4b648d4a635f4e850 span.kind=server system=grpc username=root1752gitlab # [ 321.017185] gitaly[916]: time="2026-10-02T16:57:55.927Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=75 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5101 command.oublock=8 command.real_time_ms=81 command.spawn_token_fork_ms=1 command.system_time_ms=32 command.user_time_ms=43 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:58:05.179" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:57:55.839" grpc.time_ms=87.735 pid=916 ref=refs/keep-around/4fe91ef492212e25a90e9ab4b648d4a635f4e850 remote_ip="2001:db8:1::1" revision=4fe91ef492212e25a90e9ab4b648d4a635f4e850 span.kind=server system=grpc username=root1753gitlab # [ 321.041343] gitaly[916]: time="2026-10-02T16:57:55.933Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:25.531" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:57:55.932" grpc.time_ms=0.608 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1754gitlab # [ 321.382873] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:57:56 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 8141755gitlab # [ 321.387787] nixbot[3482]: {"ts": "2026-10-02T16:57:56.320Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1756gitlab # [ 321.394253] nixbot[3482]: {"ts": "2026-10-02T16:57:56.330Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4fe91ef49221-0"}1757gitlab # [ 321.573520] nixbot[3482]: {"ts": "2026-10-02T16:57:56.510Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4fe91ef49221-0", "argv0": "bwrap"}1758gitlab # [ 321.696942] systemd[1]: Started Nix Daemon.1759gitlab # [ 321.769771] nix-daemon[3641]: accepted connection from pid 3636, user nixbot1760gitlab # [ 326.308055] nix-daemon[3641]: accepted connection from pid 3680, user nixbot1761gitlab # [ 326.321706] nix-daemon[3641]: accepted connection from pid 3636, user nixbot1762gitlab # [ 330.635302] nix-daemon[3641]: accepted connection from pid 3636, user nixbot1763gitlab # [ 330.776718] gitaly[916]: time="2026-10-02T16:58:05.712Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTDY0A2V1DNCMVPGNPWYF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:35.311" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:58:05.711" grpc.time_ms=0.891 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1764gitlab # [ 330.930367] gitaly[916]: time="2026-10-02T16:58:05.866Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRTDY0A2V1DNCMVPGNPWYF grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:58:15.195" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:58:05.855" grpc.time_ms=10.852 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1765gitlab # [ 330.990555] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:05 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2941766gitlab # [ 330.994081] nixbot[3482]: {"ts": "2026-10-02T16:58:05.928Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1767gitlab # [ 331.052876] gitaly[916]: time="2026-10-02T16:58:05.989Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTE7E0MV4TW7GKS7FBDDS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:35.588" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:58:05.988" grpc.time_ms=0.743 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1768gitlab # [ 331.105248] gitaly[916]: time="2026-10-02T16:58:06.041Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRTE7E0MV4TW7GKS7FBDDS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:58:16.001" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:58:06.031" grpc.time_ms=10.105 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1769gitlab # [ 331.170172] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 1721770gitlab # [ 331.174509] nixbot[3482]: {"ts": "2026-10-02T16:58:06.108Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1771gitlab # [ 331.254739] gitaly[916]: time="2026-10-02T16:58:06.191Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTED8ZE6JFYG67CQX7PSN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:58:36.089" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:58:06.190" grpc.time_ms=0.835 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1772gitlab # [ 331.301244] gitaly[916]: time="2026-10-02T16:58:06.237Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRTED8ZE6JFYG67CQX7PSN grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:58:16.196" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:58:06.226" grpc.time_ms=10.774 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1773gitlab # [ 331.367786] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:06 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1831774gitlab # [ 331.372394] nixbot[3482]: {"ts": "2026-10-02T16:58:06.305Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1775gitlab # [ 331.817526] gitaly[916]: time="2026-10-02T16:58:06.753Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRTEYP04JRH7T3V7MYN6Z7 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:06.741" grpc.time_ms=11.342 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1776gitlab # [ 331.839522] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:06 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 751777gitlab # [ 331.902460] gitaly[916]: time="2026-10-02T16:58:06.839Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF13G0DAAQRP4B0HE0JE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:06.826" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1778gitlab # [ 331.919580] gitaly[916]: time="2026-10-02T16:58:06.839Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF13G0DAAQRP4B0HE0JE grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:06.826" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1779gitlab # [ 331.941547] gitlab-workhorse[822]: time="2026-10-02T16:58:06Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671780gitlab # [ 331.944596] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:06 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 841781gitlab # [ 331.947555] gitaly[916]: time="2026-10-02T16:58:06.839Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF13G0DAAQRP4B0HE0JE grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:06.826" grpc.time_ms=12.774 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1782gitlab # [ 332.062466] gitaly[916]: time="2026-10-02T16:58:06.998Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRTF5Q47A9YX5D3EM97EW2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:06.988" grpc.time_ms=10.118 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1783gitlab # [ 332.085700] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:07 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 961784gitlab # [ 332.144871] gitaly[916]: time="2026-10-02T16:58:07.081Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF8SBFVP7A44684P2AHB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:07.067" pid=916 remote_ip="2001:db8:1::1" request_sha=7eb636c121e24c8525ce835f83e5adc54479211e response_bytes=67 span.kind=server system=grpc1785gitlab # [ 332.161516] gitaly[916]: time="2026-10-02T16:58:07.081Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF8SBFVP7A44684P2AHB grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:07.067" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1786gitlab # [ 332.183188] gitlab-workhorse[822]: time="2026-10-02T16:58:07Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671787gitlab # [ 332.186258] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:58:07 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 831788gitlab # [ 332.189166] gitaly[916]: time="2026-10-02T16:58:07.081Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=5 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRTF8SBFVP7A44684P2AHB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=7 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:58:07.067" grpc.time_ms=13.479 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1789gitlab # [ 357.751782] sidekiq-cluster[3364]: 2026-10-02T16:58:32.687Z pid=3364 tid=yoc INFO: {:message=>"started", :memwd_handler_class=>"Gitlab::Memory::Watchdog::Handlers::SidekiqHandler", :memwd_sleep_time_s=>3, :pid=>3364, :worker_id=>"sidekiq_0", :memwd_rss_bytes=>424865792}1790gitlab # [ 358.057065] sidekiq-cluster[3364]: W, [2026-10-02T16:58:32.993542 #3364] WARN -- sentry: `enable_tracing` is now deprecated in favor of `traces_sample_rate = 1.0`.1791gitlab # [ 358.203071] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:58:33.139Z","message":"Listening on queues [\"default,1\", \"mailers,1\"]"}1792gitlab # [ 361.366315] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:58:36.302Z","message":"GitLab reliable fetch activated!"}1793gitlab # [ 361.366796] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:58:36.302Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"internal\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1794gitlab # [ 396.888257] postgres[4276]: [4276] ERROR: relation "table_without_models" does not exist1795gitlab # [ 396.888635] postgres[4276]: [4276] STATEMENT: SELECT pg_get_serial_sequence('table_without_models', 'id')1796gitlab # [ 401.704863] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:16.640Z","message":"Booted Rails 7.2.3.1 application in production environment"}1797gitlab # [ 401.708788] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:16.645Z","message":"Running in ruby 3.3.10 (2025-10-23 revision 343ea05002) [aarch64-linux]"}1798gitlab # [ 401.712327] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:16.648Z","message":"See LICENSE and the LGPL-3.0 for licensing details."}1799gitlab # [ 401.715164] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:16.649Z","message":"Upgrade to Sidekiq Pro for more features and support: https://sidekiq.org"}1800gitlab # [ 401.718531] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:16.650Z","message":"Sidekiq 7.3.9 connecting to Redis with options {:size=\u003e6, :pool_name=\u003e\"default\", :path=\u003e\"/run/gitlab/redis.sock\", :custom=\u003e{:instrumentation_class=\u003e\"Queues\"}, :url=\u003enil}"}1801gitlab: (finished: waiting for success: journalctl -u gitlab-sidekiq _SYSTEMD_INVOCATION_ID=$(systemctl show -P InvocationID gitlab-sidekiq) | grep -q 'Booted Rails', in 82.94 seconds)1802gitlab: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master1803gitlab # [ 402.258447] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 401 343 "" "git/2.55.0" 741804gitlab # [ 402.384578] gitaly[916]: time="2026-10-02T16:59:17.320Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=580 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKTBQNHXBZT08FVNMMQH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:17.308" grpc.time_ms=11.654 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1805gitlab # Enumerating objects: 5, done.1806gitlab # [ 402.406831] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:17 +0000] "GET /root/test-flake.git/info/refs?service=git-receive-pack HTTP/1.1" 200 202 "" "git/2.55.0" 1241807gitlab # Counting objects: 20% (1/5)Counting objects: 40% (2/5)Counting objects: 60% (3/5)Counting objects: 80% (4/5)Counting objects: 100% (5/5)Counting objects: 100% (5/5), done.1808gitlab # Delta compression using up to 4 threads1809gitlab # Compressing objects: 50% (1/2)Compressing objects: 100% (2/2)Compressing objects: 100% (2/2), done.1810gitlab # Writing objects: 33% (1/3)Writing objects: 66% (2/3)Writing objects: 100% (3/3)Writing objects: 100% (3/3), 273 bytes | 136.00 KiB/s, done.1811gitlab # Total 3 (delta 1), reused 0 (delta 0), pack-reused 0 (from 0)1812gitlab # [ 402.720384] gitaly[916]: time="2026-10-02T16:59:17.656Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 command.count=1 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=302 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListAllLFSPointers grpc.request.deadline="2026-10-02T16:59:47.447" grpc.request.fullMethod=/gitaly.BlobService/ListAllLFSPointers grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=163 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.BlobService grpc.start_time="2026-10-02T16:59:17.647" grpc.time_ms=9.128 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1813gitlab # [ 402.767685] gitaly[916]: time="2026-10-02T16:59:17.704Z" level=info msg="Finished HTTP request" content_length_bytes=853 correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 duration_ms=184ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/allowed"1814gitlab # [ 402.803897] gitaly[916]: time="2026-10-02T16:59:17.740Z" level=info msg="Finished HTTP request" content_length_bytes=36 correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 duration_ms=35ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/pre_receive"1815gitlab # [ 402.811662] gitaly[916]: time="2026-10-02T16:59:17.740Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PreReceiveHook grpc.request.fullMethod=/gitaly.HookService/PreReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=5313 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:17.519" grpc.time_ms=221.577 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1816gitlab # [ 402.858733] gitaly[916]: time="2026-10-02T16:59:17.795Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=UpdateHook grpc.request.fullMethod=/gitaly.HookService/UpdateHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4783 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:17.794" grpc.time_ms=0.804 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1817gitlab # [ 402.908519] gitaly[916]: time="2026-10-02T16:59:17.845Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4782 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:17.844" grpc.time_ms=0.686 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1818gitlab # [ 402.958340] gitaly[916]: time="2026-10-02T16:59:17.894Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4784 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:17.894" grpc.time_ms=0.683 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1819gitlab # [ 403.121821] gitaly[916]: time="2026-10-02T16:59:18.058Z" level=info msg="Finished HTTP request" content_length_bytes=50 correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 duration_ms=102ns method=POST pid=916 status=200 url="http://unix/api/v4/internal/post_receive"1820gitlab # [ 403.130071] gitaly[916]: time="2026-10-02T16:59:18.064Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceiveHook grpc.request.fullMethod=/gitaly.HookService/PostReceiveHook grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=4807 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:17.954" grpc.time_ms=109.361 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1821gitlab # [ 403.151897] gitaly[916]: time="2026-10-02T16:59:18.088Z" level=info msg="diskcache state change" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 diskcache=20a843d5-e22b-45f4-8e93-2055e3eb599f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:17.451" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1822gitlab # To http://gitlab/root/test-flake.git1823gitlab # [ 403.168489] gitaly[916]: time="2026-10-02T16:59:18.089Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=234 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=851968 command.minflt=13448 command.oublock=32 command.real_time_ms=632 command.spawn_token_fork_ms=1 command.system_time_ms=133 command.user_time_ms=101 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=PostReceivePack grpc.request.fullMethod=/gitaly.SmartHTTPService/PostReceivePack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=605 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=56 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:17.451" grpc.time_ms=637.258 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc user_id=user-1 username=root1824gitlab # 4fe91ef..4a96707 master -> master1825gitlab # [ 403.190164] gitlab-workhorse[822]: time="2026-10-02T16:59:18Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-receive-pack written_bytes=521826gitlab # [ 403.193029] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:18 +0000] "POST /root/test-flake.git/git-receive-pack HTTP/1.1" 200 52 "" "git/2.55.0" 7291827gitlab: (finished: must succeed: cd /tmp/test-flake && echo '# trigger' >> flake.nix && git add flake.nix && git commit -m 'trigger build' && git push origin master, in 1.07 seconds)1828gitlab: must succeed: git -C /tmp/test-flake rev-parse master1829gitlab: (finished: must succeed: git -C /tmp/test-flake rev-parse master, in 0.02 seconds)1830gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1831gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.13 seconds)1832gitlab # [ 403.341371] nixbot[3482]: INFO: ::ffff:127.0.0.1:53814 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1833gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1834gitlab # [ 404.378696] nixbot[3482]: INFO: ::ffff:127.0.0.1:53822 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1835gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1836gitlab # [ 404.441519] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.377Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"39c4184d4f606780a4d111d3","created_at":"2026-10-02T16:57:43.849Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:43.897Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-39c4184d4f606780a4d111d3: start","job_status":"start","queue_duration_s":95.479949,"scheduling_latency_s":95.480005}1837gitlab # [ 404.636138] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.572Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Namespaces::ProcessSyncEventsWorker","jid":"39c4184d4f606780a4d111d3","created_at":"2026-10-02T16:57:43.849Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:99b01e5f7dc2a9b47ce31029252a21c1a7e02ef97b2a00a70d65582147f34ddb","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:43.897Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessSyncEventsWorker JID-39c4184d4f606780a4d111d3: done: 0.195316 sec","job_status":"done","queue_duration_s":95.479949,"scheduling_latency_s":95.480005,"redis_calls":23,"redis_duration_s":0.004991,"redis_read_bytes":910,"redis_write_bytes":2540,"redis_feature_flag_calls":13,"redis_feature_flag_duration_s":0.002471,"redis_feature_flag_read_bytes":902,"redis_feature_flag_write_bytes":1375,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001313,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":606,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001207,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":559,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":9,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.097127,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10569926,"extra.namespaces_process_sync_events_worker.estimated_total_events":1,"extra.namespaces_process_sync_events_worker.consumable_events":1,"extra.namespaces_process_sync_events_worker.processed_events":1,"duration_s":0.195316,"completed_at":"2026-10-02T16:59:19.572Z","load_balancing_strategy":"primary","db_duration_s":0.058488,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1838gitlab # [ 404.706445] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.573Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a496d6d380142a2f1e837ea4","created_at":"2026-10-02T16:57:43.914Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:43.922Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a496d6d380142a2f1e837ea4: start","job_status":"start","queue_duration_s":95.65098,"scheduling_latency_s":95.651004}1839gitlab # [ 404.748997] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.685Z","retry":25,"queue":"default","version":0,"store":null,"args":[],"class":"Projects::ProcessSyncEventsWorker","jid":"a496d6d380142a2f1e837ea4","created_at":"2026-10-02T16:57:43.914Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"cell","meta.user":"root","meta.gl_user_id":1,"meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:8f18dd753e91d7093aa53434170736240463ade652a5af40dc3edbef4e4c647e","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:43.922Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::ProcessSyncEventsWorker JID-a496d6d380142a2f1e837ea4: done: 0.111355 sec","job_status":"done","queue_duration_s":95.65098,"scheduling_latency_s":95.651004,"redis_calls":14,"redis_duration_s":0.002861,"redis_read_bytes":140,"redis_write_bytes":1653,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000569,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":422,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001324,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":680,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000968,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":551,"db_count":8,"db_write_count":2,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.08468,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0865012,"extra.projects_process_sync_events_worker.estimated_total_events":1,"extra.projects_process_sync_events_worker.consumable_events":1,"extra.projects_process_sync_events_worker.processed_events":1,"duration_s":0.111355,"completed_at":"2026-10-02T16:59:19.685Z","load_balancing_strategy":"primary","db_duration_s":0.020841,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1840gitlab # [ 404.810784] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.686Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d6e7c5c14056d849c9f10e59","created_at":"2026-10-02T16:57:44.528Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.536Z","job_size_bytes":5,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d6e7c5c14056d849c9f10e59: start","job_status":"start","queue_duration_s":95.150088,"scheduling_latency_s":95.150106}1841gitlab # [ 404.924006] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.860Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"authorized_project_update","args":["1","1"],"class":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker","jid":"d6e7c5c14056d849c9f10e59","created_at":"2026-10-02T16:57:44.528Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"permissions","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:d4fffa6b61652ecbf8fd1e3329fcdddc8f7fd5d13476a16aa4467c900fae00b7","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.536Z","job_size_bytes":5,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AuthorizedProjectUpdate::ProjectRecalculatePerUserWorker JID-d6e7c5c14056d849c9f10e59: done: 0.173441 sec","job_status":"done","queue_duration_s":95.150088,"scheduling_latency_s":95.150106,"redis_calls":12,"redis_duration_s":0.002241,"redis_read_bytes":138,"redis_write_bytes":1541,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000521,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":491,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":784,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000381,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":266,"db_count":14,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":14,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.015,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.141652,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030512,"duration_s":0.173441,"completed_at":"2026-10-02T16:59:19.860Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.026946,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1842gitlab # [ 404.994303] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:19.861Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"84f96b16476b1e804e82fa2a","created_at":"2026-10-02T16:57:44.567Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.572Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-84f96b16476b1e804e82fa2a: start","job_status":"start","queue_duration_s":95.288887,"scheduling_latency_s":95.288906}1843gitlab # [ 405.141511] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:20.077Z","retry":3,"queue":"default","version":0,"store":null,"args":["1"],"class":"Projects::PostCreationWorker","jid":"84f96b16476b1e804e82fa2a","created_at":"2026-10-02T16:57:44.567Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:115ed5a4f96bb7840d9513c0e53c69eaf78cc612f8a9b05f8b28bc5567d331ce","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.572Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Projects::PostCreationWorker JID-84f96b16476b1e804e82fa2a: done: 0.216033 sec","job_status":"done","queue_duration_s":95.288887,"scheduling_latency_s":95.288906,"redis_calls":11,"redis_duration_s":0.002528,"redis_read_bytes":137,"redis_write_bytes":1147,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000594,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":407,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001485,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":538,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000449,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":202,"db_count":35,"db_write_count":6,"db_cached_count":2,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":35,"db_replica_write_count":0,"db_primary_write_count":6,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.027,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":34,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":6,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":2,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.179,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.179,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.026,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.15152,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043762,"duration_s":0.216033,"completed_at":"2026-10-02T16:59:20.077Z","load_balancing_strategy":"primary","db_duration_s":0.067666,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1844gitlab # [ 405.209593] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:20.079Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"47b68ce99ba0bff686d828c3","created_at":"2026-10-02T16:57:44.576Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.581Z","job_size_bytes":89,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-47b68ce99ba0bff686d828c3: start","job_status":"start","queue_duration_s":95.497667,"scheduling_latency_s":95.497688}1845gitlab # [ 405.230390] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:20.117Z","retry":25,"queue":"default","version":0,"store":null,"args":["Projects::ProjectCreatedEvent","{\"project_id\"=\u003e1, \"namespace_id\"=\u003e1, \"root_namespace_id\"=\u003e1}"],"class":"Ci::InitializePipelinesIidSequenceWorker","jid":"47b68ce99ba0bff686d828c3","created_at":"2026-10-02T16:57:44.576Z","trace_propagation_headers":{"sentry-trace":"d651b410def44331be33a07373044ebf-f604eb65f9944610","baggage":"sentry-trace_id=d651b410def44331be33a07373044ebf,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSQP3AWH4A2297K8FQB5K","meta.caller_id":"POST /api/:version/projects","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:da4f511b278e5fec59a0525ff76c481fade26f467483004bc3dfa54e1876aecf","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:44.581Z","job_size_bytes":89,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::InitializePipelinesIidSequenceWorker JID-47b68ce99ba0bff686d828c3: done: 0.03805 sec","job_status":"done","queue_duration_s":95.497667,"scheduling_latency_s":95.497688,"redis_calls":11,"redis_duration_s":0.002349,"redis_read_bytes":137,"redis_write_bytes":1267,"redis_feature_flag_calls":3,"redis_feature_flag_duration_s":0.000587,"redis_feature_flag_read_bytes":130,"redis_feature_flag_write_bytes":443,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001339,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":594,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":230,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025805,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00043774,"duration_s":0.03805,"completed_at":"2026-10-02T16:59:20.117Z","load_balancing_strategy":"primary","db_duration_s":0.005304,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1846gitlab # [ 405.277485] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:20.118Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4fe91ef492212e25a90e9ab4b648d4a635f4e850 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9d597162f1fe3643f79a9a79","created_at":"2026-10-02T16:57:51.597Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-2c6f90f1fb8b4bf3","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:51.628Z","job_size_bytes":185,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9d597162f1fe3643f79a9a79: start","job_status":"start","queue_duration_s":88.490475,"scheduling_latency_s":88.490494}1847gitlab # [ 405.295042] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:20.135Z","message":"changes: \"0000000000000000000000000000000000000000 4fe91ef492212e25a90e9ab4b648d4a635f4e850 refs/heads/master\\n\""}1848gitlab # [ 405.298017] gitaly[916]: time="2026-10-02T16:59:20.174Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:30.002" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:20.162" grpc.time_ms=11.08 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1849gitlab # [ 405.311814] gitaly[916]: time="2026-10-02T16:59:20.191Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:59:30.020" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:20.180" grpc.time_ms=9.97 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1850gitlab # [ 405.325866] gitaly[916]: time="2026-10-02T16:59:20.262Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:59:30.089" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:20.250" grpc.time_ms=12.121 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1851gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1852gitlab # [ 405.372479] gitaly[916]: time="2026-10-02T16:59:20.309Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3935 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:20.308" grpc.time_ms=0.512 pid=916 span.kind=server system=grpc1853gitlab # [ 405.405397] gitaly[916]: time="2026-10-02T16:59:20.341Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3937 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:20.341" grpc.time_ms=0.578 pid=916 span.kind=server system=grpc1854gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1855gitlab # [ 405.421530] gitaly[916]: time="2026-10-02T16:59:20.353Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q diskcache=533dfe75-cb4c-40bd-8a3e-954921471c76 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:59:30.114" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:20.274" pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1856gitlab # [ 405.437597] gitaly[916]: time="2026-10-02T16:59:20.353Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=77 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5020 command.oublock=8 command.real_time_ms=74 command.spawn_token_fork_ms=0 command.system_time_ms=36 command.user_time_ms=41 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:59:30.114" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=148 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:20.274" grpc.time_ms=78.565 pid=916 ref=HEAD remote_ip="2001:db8:1::1" revision=refs/heads/master span.kind=server system=grpc username=root1857gitlab # [ 405.456774] nixbot[3482]: INFO: ::ffff:127.0.0.1:53836 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1858gitlab # [ 405.459210] gitaly[916]: time="2026-10-02T16:59:20.374Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryExists grpc.request.deadline="2026-10-02T16:59:30.213" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:20.374" grpc.time_ms=0.217 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1859gitlab # [ 405.471681] gitaly[916]: time="2026-10-02T16:59:20.393Z" level=info msg="finished unary call with code OK" command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=859 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=0 command.system_time_ms=11 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindDefaultBranchName grpc.request.deadline="2026-10-02T16:59:30.218" grpc.request.fullMethod=/gitaly.RefService/FindDefaultBranchName grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=19 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:20.379" grpc.time_ms=14.397 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1860gitlab # [ 405.563044] gitaly[916]: time="2026-10-02T16:59:20.499Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=314 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindCommits grpc.request.deadline="2026-10-02T16:59:50.091" grpc.request.fullMethod=/gitaly.CommitService/FindCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=173 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:20.491" grpc.time_ms=7.531 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1861gitlab # [ 405.601612] gitaly[916]: time="2026-10-02T16:59:20.538Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:59:30.007" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:20.537" grpc.time_ms=0.36 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1862gitlab # [ 405.618679] gitaly[916]: time="2026-10-02T16:59:20.547Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=316 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:59:30.010" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=112 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:59:20.541" grpc.time_ms=5.976 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1863gitlab # [ 405.639399] gitaly[916]: time="2026-10-02T16:59:20.574Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:30.034" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:20.564" grpc.time_ms=10.348 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1864gitlab # [ 405.656660] gitaly[916]: time="2026-10-02T16:59:20.591Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:30.050" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:20.580" grpc.time_ms=10.218 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1865gitlab # [ 405.686465] gitaly[916]: time="2026-10-02T16:59:20.623Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=12 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:59:50.205" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=155 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=80 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:20.605" grpc.time_ms=17.26 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1866gitlab # [ 406.248404] gitaly[916]: time="2026-10-02T16:59:21.185Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:59:31.022" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:21.183" grpc.time_ms=1.592 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1867gitlab # [ 406.292629] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.228Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","0000000000000000000000000000000000000000 4fe91ef492212e25a90e9ab4b648d4a635f4e850 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"9d597162f1fe3643f79a9a79","created_at":"2026-10-02T16:57:51.597Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-2c6f90f1fb8b4bf3","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:57:51.628Z","job_size_bytes":185,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-9d597162f1fe3643f79a9a79: done: 1.109272 sec","job_status":"done","queue_duration_s":88.490475,"scheduling_latency_s":88.490494,"gitaly_calls":13,"gitaly_duration_s":0.220706,"redis_calls":198,"redis_allowed_cross_slot_calls":6,"redis_duration_s":0.043077,"redis_read_bytes":2103,"redis_write_bytes":21746,"redis_cache_calls":15,"redis_cache_duration_s":0.003784,"redis_cache_read_bytes":20,"redis_cache_write_bytes":991,"redis_feature_flag_calls":27,"redis_feature_flag_duration_s":0.005967,"redis_feature_flag_read_bytes":1678,"redis_feature_flag_write_bytes":3082,"redis_queues_calls":10,"redis_queues_duration_s":0.001705,"redis_queues_read_bytes":10,"redis_queues_write_bytes":7834,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.003103,"redis_queues_metadata_read_bytes":13,"redis_queues_metadata_write_bytes":1467,"redis_repository_cache_calls":90,"redis_repository_cache_allowed_cross_slot_calls":6,"redis_repository_cache_duration_s":0.017401,"redis_repository_cache_read_bytes":249,"redis_repository_cache_write_bytes":6237,"redis_shared_state_calls":46,"redis_shared_state_duration_s":0.011117,"redis_shared_state_read_bytes":133,"redis_shared_state_write_bytes":2135,"db_count":47,"db_write_count":8,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":47,"db_replica_write_count":0,"db_primary_write_count":8,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.057,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":47,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":8,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.277,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.307,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.057,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.795123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00065422,"exclusive_lock_hold_duration_s":0.03037512,"duration_s":1.109272,"completed_at":"2026-10-02T16:59:21.227Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.099914,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1868gitlab # [ 406.353960] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.230Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7e85a49e728592a6e55c9a8c","created_at":"2026-10-02T16:57:55.942Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:55.972Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7e85a49e728592a6e55c9a8c: start","job_status":"start","queue_duration_s":85.257133,"scheduling_latency_s":85.257164}1869gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1870gitlab # [ 406.406231] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.342Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"7e85a49e728592a6e55c9a8c","created_at":"2026-10-02T16:57:55.942Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:efbbb938170501c99cc41a8a2b4ae8ea32bc3fc95221eec9a69e494b4dc14b10","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:55.972Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-7e85a49e728592a6e55c9a8c: done: 0.112187 sec","job_status":"done","queue_duration_s":85.257133,"scheduling_latency_s":85.257164,"redis_calls":16,"redis_duration_s":0.004021,"redis_read_bytes":269,"redis_write_bytes":1850,"redis_feature_flag_calls":8,"redis_feature_flag_duration_s":0.002106,"redis_feature_flag_read_bytes":262,"redis_feature_flag_write_bytes":1032,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001376,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":590,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000539,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":228,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079646,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038376,"duration_s":0.112187,"completed_at":"2026-10-02T16:59:21.342Z","load_balancing_strategy":"primary","db_duration_s":0.024505,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1871gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1872gitlab # [ 406.461388] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.344Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ab17ef4d328f41b298ae0e78","created_at":"2026-10-02T16:57:55.973Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:55.980Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ab17ef4d328f41b298ae0e78: start","job_status":"start","queue_duration_s":85.363828,"scheduling_latency_s":85.363853}1873gitlab # [ 406.481018] nixbot[3482]: INFO: ::ffff:127.0.0.1:53850 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1874gitlab # [ 406.532170] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.468Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e1, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ab17ef4d328f41b298ae0e78","created_at":"2026-10-02T16:57:55.973Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:98ff20ecdaa20afd17480d536c184377f85c5a4d32875b1023fc943dfe20026b","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:55.980Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ab17ef4d328f41b298ae0e78: done: 0.1239 sec","job_status":"done","queue_duration_s":85.363828,"scheduling_latency_s":85.363853,"redis_calls":21,"redis_duration_s":0.0052,"redis_read_bytes":148,"redis_write_bytes":2264,"redis_feature_flag_calls":9,"redis_feature_flag_duration_s":0.002654,"redis_feature_flag_read_bytes":136,"redis_feature_flag_write_bytes":1264,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001344,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":578,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001202,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":422,"db_count":15,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.029,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.021,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081479,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00045054,"duration_s":0.1239,"completed_at":"2026-10-02T16:59:21.468Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.033655,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1875gitlab # [ 406.599958] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:21.470Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"eccc8bfb0724156de5c3d7ca","created_at":"2026-10-02T16:57:56.288Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:56.295Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-eccc8bfb0724156de5c3d7ca: start","job_status":"start","queue_duration_s":85.174294,"scheduling_latency_s":85.174316}1876gitlab # [ 406.868357] gitaly[916]: time="2026-10-02T16:59:21.804Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:21.803" grpc.time_ms=0.519 pid=916 span.kind=server system=grpc1877gitlab # [ 406.903250] gitaly[916]: time="2026-10-02T16:59:21.839Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:21.839" grpc.time_ms=0.555 pid=916 span.kind=server system=grpc1878gitlab # [ 406.938512] gitaly[916]: time="2026-10-02T16:59:21.875Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:21.874" grpc.time_ms=0.587 pid=916 span.kind=server system=grpc1879gitlab # [ 406.954590] gitaly[916]: time="2026-10-02T16:59:21.883Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 diskcache=1098d330-37a7-4b87-b792-0f67e05a3d3b grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:59:51.368" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:21.769" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1880gitlab # [ 406.971148] gitaly[916]: time="2026-10-02T16:59:21.884Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=117 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7442 command.oublock=0 command.real_time_ms=110 command.spawn_token_fork_ms=0 command.system_time_ms=65 command.user_time_ms=52 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:59:51.368" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:21.769" grpc.time_ms=114.528 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1881gitlab # [ 407.099405] gitaly[916]: time="2026-10-02T16:59:22.036Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:59:52.034" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.034" grpc.time_ms=1.198 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1882gitlab # [ 407.150402] gitaly[916]: time="2026-10-02T16:59:22.086Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRT41F6BRQJ0WGYB6DPGJ3 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:52.085" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.085" grpc.time_ms=0.977 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1883gitlab # [ 407.175592] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.111Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"eccc8bfb0724156de5c3d7ca","created_at":"2026-10-02T16:57:56.288Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-7b838f0a67de4154","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:57:56.295Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-eccc8bfb0724156de5c3d7ca: done: 0.641532 sec","job_status":"done","queue_duration_s":85.174294,"scheduling_latency_s":85.174316,"gitaly_calls":3,"gitaly_duration_s":0.125409,"redis_calls":117,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.026227,"redis_read_bytes":455,"redis_write_bytes":28004,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001313,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.002303,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_feature_flag_calls":40,"redis_feature_flag_duration_s":0.008519,"redis_feature_flag_read_bytes":294,"redis_feature_flag_write_bytes":6581,"redis_queues_calls":25,"redis_queues_duration_s":0.004463,"redis_queues_read_bytes":34,"redis_queues_write_bytes":14498,"redis_queues_metadata_calls":21,"redis_queues_metadata_duration_s":0.005121,"redis_queues_metadata_read_bytes":32,"redis_queues_metadata_write_bytes":3370,"redis_rate_limiting_calls":2,"redis_rate_limiting_duration_s":0.000662,"redis_rate_limiting_read_bytes":2,"redis_rate_limiting_write_bytes":166,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.0005,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002033,"redis_shared_state_read_bytes":11,"redis_shared_state_write_bytes":851,"db_count":52,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":52,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.061,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":35,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.025,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.029,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.031,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.438347,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.5757331,"duration_s":0.641532,"completed_at":"2026-10-02T16:59:22.111Z","load_balancing_strategy":"primary","db_duration_s":0.072744,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1884gitlab # [ 407.250339] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.113Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4fe91ef492212e25a90e9ab4b648d4a635f4e850 4a967074777ce7606287d14f5463040267a12c45 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7af246eeb7e1abfb8b80fd96","created_at":"2026-10-02T16:59:18.022Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-37087f3d052e4033","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:18.029Z","job_size_bytes":185,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7af246eeb7e1abfb8b80fd96: start","job_status":"start","queue_duration_s":4.083953,"scheduling_latency_s":4.08398}1885gitlab # [ 407.269962] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.130Z","message":"changes: \"4fe91ef492212e25a90e9ab4b648d4a635f4e850 4a967074777ce7606287d14f5463040267a12c45 refs/heads/master\\n\""}1886gitlab # [ 407.272614] gitaly[916]: time="2026-10-02T16:59:22.165Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:32.004" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=136 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=63 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:22.154" grpc.time_ms=11.327 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1887gitlab # [ 407.286241] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.186Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"814a5b3d5edd0153d4d5a561","created_at":"2026-10-02T16:59:22.184Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","duplicate-of":"46c36cbd972353241bc8d46f","job_size_bytes":46,"pid":3364,"job_status":"deduplicated","message":"ProjectCacheWorker JID-814a5b3d5edd0153d4d5a561: deduplicated: until executing","deduplication.type":"until executing"}1888gitlab # [ 407.301525] gitaly[916]: time="2026-10-02T16:59:22.181Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=HasLocalBranches grpc.request.deadline="2026-10-02T16:59:32.011" grpc.request.fullMethod=/gitaly.RepositoryService/HasLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:22.171" grpc.time_ms=10.099 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1889gitlab # [ 407.315411] gitaly[916]: time="2026-10-02T16:59:22.214Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=FindLocalBranches grpc.request.deadline="2026-10-02T16:59:32.043" grpc.request.fullMethod=/gitaly.RefService/FindLocalBranches grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=266 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:22.203" grpc.time_ms=10.837 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1890gitlab # [ 407.329443] gitaly[916]: time="2026-10-02T16:59:22.231Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=597 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommits grpc.request.deadline="2026-10-02T16:59:52.219" grpc.request.fullMethod=/gitaly.CommitService/ListCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=212 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.219" grpc.time_ms=11.713 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1891gitlab # [ 407.345245] gitaly[916]: time="2026-10-02T16:59:22.251Z" level=info msg="finished streaming call with code OK" command.count=1 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=318 command.oublock=0 command.real_time_ms=5 command.spawn_token_fork_ms=0 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=CommitDelta grpc.request.deadline="2026-10-02T16:59:32.084" grpc.request.fullMethod=/gitaly.DiffService/CommitDelta grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=207 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=116 grpc.service=gitaly.DiffService grpc.start_time="2026-10-02T16:59:22.245" grpc.time_ms=6.726 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1892gitlab # [ 407.358942] gitaly[916]: time="2026-10-02T16:59:22.272Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:32.103" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:22.263" grpc.time_ms=9.328 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1893gitlab # [ 407.452454] gitaly[916]: time="2026-10-02T16:59:22.388Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=FilterShasWithSignatures grpc.request.deadline="2026-10-02T16:59:32.227" grpc.request.fullMethod=/gitaly.CommitService/FilterShasWithSignatures grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.387" grpc.time_ms=1.375 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1894gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1895gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1896gitlab # [ 407.481684] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.416Z","retry":3,"queue":"default","version":0,"store":null,"args":["project-1","user-1","4fe91ef492212e25a90e9ab4b648d4a635f4e850 4a967074777ce7606287d14f5463040267a12c45 refs/heads/master\n","{}","[FILTERED]"],"class":"Repositories::PostReceiveWorker","jid":"7af246eeb7e1abfb8b80fd96","created_at":"2026-10-02T16:59:18.022Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-37087f3d052e4033","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"POST /api/:version/internal/post_receive","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:18.029Z","job_size_bytes":185,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Repositories::PostReceiveWorker JID-7af246eeb7e1abfb8b80fd96: done: 0.302474 sec","job_status":"done","queue_duration_s":4.083953,"scheduling_latency_s":4.08398,"gitaly_calls":7,"gitaly_duration_s":0.081548,"redis_calls":110,"redis_allowed_cross_slot_calls":4,"redis_duration_s":0.023788,"redis_read_bytes":501,"redis_write_bytes":11941,"redis_cache_calls":9,"redis_cache_duration_s":0.00376,"redis_cache_read_bytes":14,"redis_cache_write_bytes":593,"redis_queues_calls":6,"redis_queues_duration_s":0.000997,"redis_queues_read_bytes":9,"redis_queues_write_bytes":5718,"redis_queues_metadata_calls":10,"redis_queues_metadata_duration_s":0.002576,"redis_queues_metadata_read_bytes":163,"redis_queues_metadata_write_bytes":1428,"redis_repository_cache_calls":41,"redis_repository_cache_allowed_cross_slot_calls":4,"redis_repository_cache_duration_s":0.007067,"redis_repository_cache_read_bytes":176,"redis_repository_cache_write_bytes":2193,"redis_shared_state_calls":44,"redis_shared_state_duration_s":0.009388,"redis_shared_state_read_bytes":139,"redis_shared_state_write_bytes":2009,"db_count":28,"db_write_count":3,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":28,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.024,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":28,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.031,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.031,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.024,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.195086,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00087994,"exclusive_lock_hold_duration_s":0.02387166,"duration_s":0.302474,"completed_at":"2026-10-02T16:59:22.415Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.024841,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1897gitlab # [ 407.553665] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.420Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"46c36cbd972353241bc8d46f","created_at":"2026-10-02T16:59:20.194Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.218Z","job_size_bytes":46,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-46c36cbd972353241bc8d46f: start","job_status":"start","queue_duration_s":2.20199,"scheduling_latency_s":2.20202}1898gitlab # [ 407.571439] gitaly[916]: time="2026-10-02T16:59:22.484Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RepositoryInfo grpc.request.deadline="2026-10-02T22:59:22.478" grpc.request.fullMethod=/gitaly.RepositoryService/RepositoryInfo grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:22.478" grpc.time_ms=5.387 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1899gitlab # [ 407.583207] nixbot[3482]: INFO: ::ffff:127.0.0.1:53860 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1900gitlab # [ 407.586179] gitaly[916]: time="2026-10-02T16:59:22.504Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=595 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:59:52.094" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.495" grpc.time_ms=9.431 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1901gitlab # [ 407.605060] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.541Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"repository_size\", \"commit_count\"]","true"],"class":"ProjectCacheWorker","jid":"46c36cbd972353241bc8d46f","created_at":"2026-10-02T16:59:20.194Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6f67506c84ed98a5106f7d9419d8e0f06d78859d539bcf9f821535b1741756e6","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.218Z","job_size_bytes":46,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-46c36cbd972353241bc8d46f: done: 0.120924 sec","job_status":"done","queue_duration_s":2.20199,"scheduling_latency_s":2.20202,"gitaly_calls":2,"gitaly_duration_s":0.02005,"redis_calls":30,"redis_allowed_cross_slot_calls":2,"redis_duration_s":0.007279,"redis_read_bytes":69,"redis_write_bytes":4884,"redis_feature_flag_calls":6,"redis_feature_flag_duration_s":0.001435,"redis_feature_flag_read_bytes":6,"redis_feature_flag_write_bytes":1005,"redis_queues_calls":3,"redis_queues_duration_s":0.000573,"redis_queues_read_bytes":4,"redis_queues_write_bytes":2112,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001836,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":710,"redis_repository_cache_calls":11,"redis_repository_cache_allowed_cross_slot_calls":2,"redis_repository_cache_duration_s":0.002685,"redis_repository_cache_read_bytes":48,"redis_repository_cache_write_bytes":753,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.00075,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":304,"db_count":10,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":10,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":10,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":2,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.079091,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00036222,"duration_s":0.120924,"completed_at":"2026-10-02T16:59:22.540Z","load_balancing_strategy":"primary","db_duration_s":0.014117,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1902gitlab # [ 407.655698] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.543Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b692c109d82ba28cce5138b9","created_at":"2026-10-02T16:59:20.238Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.245Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b692c109d82ba28cce5138b9: start","job_status":"start","queue_duration_s":2.297686,"scheduling_latency_s":2.297714}1903gitlab # [ 407.673317] gitaly[916]: time="2026-10-02T16:59:22.599Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q diskcache=e4e2c73b-de5c-40d9-a178-d0395df67f35 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:59:22.574" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.575" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1904gitlab # [ 407.683648] gitaly[916]: time="2026-10-02T16:59:22.599Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=4 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1172 command.oublock=0 command.real_time_ms=29 command.spawn_token_fork_ms=0 command.system_time_ms=14 command.user_time_ms=2 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:59:22.574" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.575" grpc.time_ms=24.427 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1905gitlab # [ 407.714806] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.650Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"b692c109d82ba28cce5138b9","created_at":"2026-10-02T16:59:20.238Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.245Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-b692c109d82ba28cce5138b9: done: 0.107873 sec","job_status":"done","queue_duration_s":2.297686,"scheduling_latency_s":2.297714,"gitaly_calls":1,"gitaly_duration_s":0.027409,"redis_calls":15,"redis_duration_s":0.0035050000000000003,"redis_read_bytes":68,"redis_write_bytes":1520,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000416,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":305,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001342,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":558,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000595,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001152,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":520,"db_count":12,"db_write_count":4,"db_cached_count":0,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":4,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.01,"db_main_txn_count":2,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":4,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.013,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.025,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.01,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.055648,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.07496806,"duration_s":0.107873,"completed_at":"2026-10-02T16:59:22.650Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.017792,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1906gitlab # [ 407.763561] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.652Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f59094518370f1f295b9a93f","created_at":"2026-10-02T16:59:20.368Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.371Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f59094518370f1f295b9a93f: start","job_status":"start","queue_duration_s":2.281729,"scheduling_latency_s":2.281754}1907gitlab # [ 407.779649] gitaly[916]: time="2026-10-02T16:59:22.695Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=596 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CountCommits grpc.request.deadline="2026-10-02T16:59:52.285" grpc.request.fullMethod=/gitaly.CommitService/CountCommits grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.685" grpc.time_ms=9.63 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1908gitlab # [ 407.794364] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.706Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[]","[\"commit_count\"]"],"class":"ProjectCacheWorker","jid":"f59094518370f1f295b9a93f","created_at":"2026-10-02T16:59:20.368Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:9a580f4ed3b98848550cd9d8a9e00b9c6cdb806321df08c03ad3e2050bcedc4f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.371Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ProjectCacheWorker JID-f59094518370f1f295b9a93f: done: 0.053479 sec","job_status":"done","queue_duration_s":2.281729,"scheduling_latency_s":2.281754,"gitaly_calls":1,"gitaly_duration_s":0.012141,"redis_calls":16,"redis_duration_s":0.003336,"redis_read_bytes":52,"redis_write_bytes":2049,"redis_queues_calls":1,"redis_queues_duration_s":0.000325,"redis_queues_read_bytes":1,"redis_queues_write_bytes":980,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000958,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":437,"redis_repository_cache_calls":7,"redis_repository_cache_duration_s":0.001411,"redis_repository_cache_read_bytes":44,"redis_repository_cache_write_bytes":344,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000642,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":288,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033111,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.053479,"completed_at":"2026-10-02T16:59:22.706Z","load_balancing_strategy":"primary","db_duration_s":0.003707,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1909gitlab # [ 407.841501] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:22.707Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a13d0d1156b128fec070bb44","created_at":"2026-10-02T16:59:20.977Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:20a9e0c3ae7b2529a32042f97fc86e5dc57e578c223bd62211cf6d9131d12d95","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.986Z","job_size_bytes":467,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a13d0d1156b128fec070bb44: start","job_status":"start","queue_duration_s":1.721794,"scheduling_latency_s":1.721817}1910gitlab # [ 407.859482] gitaly[916]: time="2026-10-02T16:59:22.738Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:52.337" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.737" grpc.time_ms=0.666 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1911gitlab # [ 407.872945] gitaly[916]: time="2026-10-02T16:59:22.750Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=7 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:59:32.210" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:22.740" grpc.time_ms=9.642 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1912gitlab # [ 407.961023] gitaly[916]: time="2026-10-02T16:59:22.897Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:59:52.496" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.896" grpc.time_ms=0.973 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1913gitlab # [ 407.974590] gitaly[916]: time="2026-10-02T16:59:22.905Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=38 command.oublock=0 command.real_time_ms=1 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:59:52.502" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:22.902" grpc.time_ms=2.735 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1914gitlab # [ 408.092597] gitaly[916]: time="2026-10-02T16:59:23.029Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:33.008" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:23.018" grpc.time_ms=10.106 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1915gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1916gitlab # [ 408.547774] nixbot[3482]: INFO: ::ffff:127.0.0.1:53876 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1917gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1918gitlab # [ 408.884219] gitaly[916]: time="2026-10-02T16:59:23.820Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=13 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=882 command.oublock=0 command.real_time_ms=14 command.spawn_token_fork_ms=0 command.system_time_ms=0 command.user_time_ms=13 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:59:53.801" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:23.802" grpc.time_ms=17.854 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1919gitlab # [ 408.908717] gitaly[916]: time="2026-10-02T16:59:23.839Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRSZJ7E0R35JZ638W8XS7Q grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:59:53.822" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:23.823" grpc.time_ms=16.68 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1920gitlab # [ 408.936297] sidekiq-cluster[3364]: {"severity":"WARN","time":"2026-10-02T16:59:23.872Z","class":"CreatePipelineWorker","correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"0000000000000000000000000000000000000000","after":"4fe91ef492212e25a90e9ab4b648d4a635f4e850","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4fe91ef492212e25a90e9ab4b648d4a635f4e850","ref":"refs/heads/master"},"retry":0}1921gitlab # [ 408.947806] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:23.880Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"a13d0d1156b128fec070bb44","created_at":"2026-10-02T16:59:20.977Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:20a9e0c3ae7b2529a32042f97fc86e5dc57e578c223bd62211cf6d9131d12d95","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:20.986Z","job_size_bytes":467,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-a13d0d1156b128fec070bb44: done: 1.172507 sec","job_status":"done","queue_duration_s":1.721794,"scheduling_latency_s":1.721817,"gitaly_calls":7,"gitaly_duration_s":0.075307,"redis_calls":51,"redis_duration_s":0.010668,"redis_read_bytes":169,"redis_write_bytes":4168,"redis_cache_calls":4,"redis_cache_duration_s":0.000883,"redis_cache_read_bytes":10,"redis_cache_write_bytes":343,"redis_feature_flag_calls":16,"redis_feature_flag_duration_s":0.005144,"redis_feature_flag_read_bytes":16,"redis_feature_flag_write_bytes":2174,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001212,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":510,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.002424,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001005,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":513,"db_count":36,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":36,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.032,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":32,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.028,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":1.045835,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031796,"duration_s":1.172507,"completed_at":"2026-10-02T16:59:23.880Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.055476,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1922gitlab # [ 409.014876] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:23.882Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"5320f727-530f-4269-a914-221d57ec2b1a\"}"],"class":"WebHookWorker","jid":"ccaae22078733d5de46cfa16","created_at":"2026-10-02T16:59:21.148Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bf3460fd030a6587e985ce1876c0351b15999a1da56a8d158bcf51e61e659c8a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.154Z","job_size_bytes":1751,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ccaae22078733d5de46cfa16: start","job_status":"start","queue_duration_s":2.727764,"scheduling_latency_s":2.727785}1923gitlab # [ 409.137418] nixbot[3482]: INFO: ::ffff:127.0.0.1:53878 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted1924gitlab # [ 409.164173] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.100Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"5320f727-530f-4269-a914-221d57ec2b1a\"}"],"class":"WebHookWorker","jid":"ccaae22078733d5de46cfa16","created_at":"2026-10-02T16:59:21.148Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-22e2a78bd99f4725","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bf3460fd030a6587e985ce1876c0351b15999a1da56a8d158bcf51e61e659c8a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.154Z","job_size_bytes":1751,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-ccaae22078733d5de46cfa16: done: 0.217459 sec","job_status":"done","queue_duration_s":2.727764,"scheduling_latency_s":2.727785,"redis_calls":19,"redis_duration_s":0.0048839999999999995,"redis_read_bytes":32,"redis_write_bytes":5367,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.001251,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":622,"redis_queues_calls":2,"redis_queues_duration_s":0.000487,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3729,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002039,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":690,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001107,"redis_shared_state_read_bytes":19,"redis_shared_state_write_bytes":326,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.06826991999997745,"cpu_s":0.131007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.0004005,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.217459,"completed_at":"2026-10-02T16:59:24.099Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004215,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1925gitlab # [ 409.238021] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.102Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e56d7a9026586d3132efd6f8","created_at":"2026-10-02T16:59:21.750Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.756Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e56d7a9026586d3132efd6f8: start","job_status":"start","queue_duration_s":2.345863,"scheduling_latency_s":2.34589}1926gitlab # [ 409.256802] gitaly[916]: time="2026-10-02T16:59:24.182Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=12 command.spawn_token_fork_ms=1 command.system_time_ms=4 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTJ734SH7M7J23F9NC07 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.168" grpc.time_ms=14.092 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1927gitlab # [ 409.272381] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:24 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 791928gitlab # [ 409.304224] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.240Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"e56d7a9026586d3132efd6f8","created_at":"2026-10-02T16:59:21.750Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.756Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-e56d7a9026586d3132efd6f8: done: 0.13807 sec","job_status":"done","queue_duration_s":2.345863,"scheduling_latency_s":2.34589,"redis_calls":10,"redis_duration_s":0.00227,"redis_read_bytes":9,"redis_write_bytes":1029,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000451,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001338,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":534,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000481,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":200,"db_count":15,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":15,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":11,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.007,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.007,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.012,"db_ci_duration_s":0.008,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.103705,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00040032,"duration_s":0.13807,"completed_at":"2026-10-02T16:59:24.240Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.027855,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}1929gitlab # [ 409.367196] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.242Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8fef869f7208f1b3c1148968","created_at":"2026-10-02T16:59:21.891Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.898Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8fef869f7208f1b3c1148968: start","job_status":"start","queue_duration_s":2.344039,"scheduling_latency_s":2.344066}1930gitlab # [ 409.385328] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.290Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"8fef869f7208f1b3c1148968","created_at":"2026-10-02T16:59:21.891Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.898Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-8fef869f7208f1b3c1148968: done: 0.047888 sec","job_status":"done","queue_duration_s":2.344039,"scheduling_latency_s":2.344066,"redis_calls":10,"redis_duration_s":0.002644,"redis_read_bytes":9,"redis_write_bytes":1165,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000539,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":602,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000509,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":234,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033156,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00034546,"duration_s":0.047888,"completed_at":"2026-10-02T16:59:24.290Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005338,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1931gitlab # [ 409.434297] gitaly[916]: time="2026-10-02T16:59:24.253Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTMSVRNYYHM6CNN0J322 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.242" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1932gitlab # [ 409.445246] gitlab-workhorse[822]: time="2026-10-02T16:59:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671933gitlab # [ 409.447171] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 681934gitlab # [ 409.449175] gitlab-workhorse[822]: time="2026-10-02T16:59:24Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=3371935gitlab # [ 409.451155] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:24 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 337 "" "git/2.55.0" 841936gitlab # [ 409.453186] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.292Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c2480e22c17d2e6119d61611","created_at":"2026-10-02T16:59:21.916Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.925Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c2480e22c17d2e6119d61611: start","job_status":"start","queue_duration_s":2.366227,"scheduling_latency_s":2.366247}1937gitlab # [ 409.470588] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.346Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"c2480e22c17d2e6119d61611","created_at":"2026-10-02T16:59:21.916Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.925Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-c2480e22c17d2e6119d61611: done: 0.053835 sec","job_status":"done","queue_duration_s":2.366227,"scheduling_latency_s":2.366247,"redis_calls":10,"redis_duration_s":0.00262,"redis_read_bytes":9,"redis_write_bytes":1205,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000419,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001381,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":622,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00082,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":244,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033257,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00031352,"duration_s":0.053835,"completed_at":"2026-10-02T16:59:24.345Z","load_balancing_strategy":"primary","db_duration_s":0.013227,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1938gitlab # [ 409.516000] gitaly[916]: time="2026-10-02T16:59:24.253Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTMSVRNYYHM6CNN0J322 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.242" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1939gitlab # [ 409.529908] nixbot[3482]: {"ts": "2026-10-02T16:59:24.381Z", "level": "info", "logger": "nixbot.build_reuse", "msg": "reusing build for tree hash", "build_id": 1, "tree_hash": "a8c90de643fa3b1e78fc9dc27f18f04a936ba912"}1940gitlab # [ 409.532551] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.347Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fe332b5d7f87d53d084bd191","created_at":"2026-10-02T16:59:21.945Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.952Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fe332b5d7f87d53d084bd191: start","job_status":"start","queue_duration_s":2.395156,"scheduling_latency_s":2.395175}1941gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1942gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.04 seconds)1943gitlab # [ 409.548750] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.409Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"fe332b5d7f87d53d084bd191","created_at":"2026-10-02T16:59:21.945Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.952Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-fe332b5d7f87d53d084bd191: done: 0.061881 sec","job_status":"done","queue_duration_s":2.395156,"scheduling_latency_s":2.395175,"redis_calls":13,"redis_duration_s":0.0076300000000000005,"redis_read_bytes":66,"redis_write_bytes":1182,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000434,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00617,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000504,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000522,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.01,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.03602,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00030346,"duration_s":0.061881,"completed_at":"2026-10-02T16:59:24.409Z","load_balancing_strategy":"primary","db_duration_s":0.012646,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1944gitlab # [ 409.595635] gitaly[916]: time="2026-10-02T16:59:24.253Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=585 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTMSVRNYYHM6CNN0J322 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.242" grpc.time_ms=10.695 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1945gitlab # [ 409.610824] nixbot[3482]: INFO: ::ffff:127.0.0.1:53894 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1946gitlab # [ 409.612889] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.411Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2c817d97cf20f3a0fa7c8a0e","created_at":"2026-10-02T16:59:21.953Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.959Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2c817d97cf20f3a0fa7c8a0e: start","job_status":"start","queue_duration_s":2.451665,"scheduling_latency_s":2.451688}1947gitlab # [ 409.629926] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.454Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"2c817d97cf20f3a0fa7c8a0e","created_at":"2026-10-02T16:59:21.953Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.959Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-2c817d97cf20f3a0fa7c8a0e: done: 0.043724 sec","job_status":"done","queue_duration_s":2.451665,"scheduling_latency_s":2.451688,"redis_calls":10,"redis_duration_s":0.002653,"redis_read_bytes":9,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000899,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":329,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001317,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":598,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000437,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":232,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.028625,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00033744,"duration_s":0.043724,"completed_at":"2026-10-02T16:59:24.454Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007625,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1948gitlab # [ 409.677242] gitaly[916]: time="2026-10-02T16:59:24.339Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTQ0QB9Y3XX266KWYP3F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.316" pid=916 remote_ip="2001:db8:1::1" request_sha=5ece69e4c395514325310b039a9bca509114388a response_bytes=337 span.kind=server system=grpc1949gitlab # [ 409.687705] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.456Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3fe6973b41658cd193a40840","created_at":"2026-10-02T16:59:21.960Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.967Z","job_size_bytes":19,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3fe6973b41658cd193a40840: start","job_status":"start","queue_duration_s":2.48889,"scheduling_latency_s":2.488912}1950gitlab # [ 409.704767] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.491Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"3fe6973b41658cd193a40840","created_at":"2026-10-02T16:59:21.960Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.967Z","job_size_bytes":19,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-3fe6973b41658cd193a40840: done: 0.035455 sec","job_status":"done","queue_duration_s":2.48889,"scheduling_latency_s":2.488912,"redis_calls":11,"redis_duration_s":0.002152,"redis_read_bytes":10,"redis_write_bytes":1121,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000452,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":287,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001241,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":640,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000459,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":194,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023068,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00029624,"duration_s":0.035455,"completed_at":"2026-10-02T16:59:24.491Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004401,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1951gitlab # [ 409.750276] nixbot[3482]: {"ts": "2026-10-02T16:59:24.600Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1952gitlab # [ 409.753181] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 657 "" "python-httpx/0.28.1" 2011953gitlab # [ 409.755771] gitaly[916]: time="2026-10-02T16:59:24.339Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTQ0QB9Y3XX266KWYP3F grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.316" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1954gitlab # [ 409.770260] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.493Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"084d7e396e79ea0fddec603c","created_at":"2026-10-02T16:59:21.971Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.977Z","job_size_bytes":66,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-084d7e396e79ea0fddec603c: start","job_status":"start","queue_duration_s":2.51579,"scheduling_latency_s":2.51581}1955gitlab # [ 409.788157] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.554Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"084d7e396e79ea0fddec603c","created_at":"2026-10-02T16:59:21.971Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.977Z","job_size_bytes":66,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-084d7e396e79ea0fddec603c: done: 0.060955 sec","job_status":"done","queue_duration_s":2.51579,"scheduling_latency_s":2.51581,"redis_calls":10,"redis_duration_s":0.010329,"redis_read_bytes":9,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003083,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":335,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.006681,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":614,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000565,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":240,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00038126,"duration_s":0.060955,"completed_at":"2026-10-02T16:59:24.554Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.010283,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1956gitlab # [ 409.835019] gitaly[916]: time="2026-10-02T16:59:24.339Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=16 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1127 command.oublock=0 command.real_time_ms=19 command.spawn_token_fork_ms=0 command.system_time_ms=16 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTQ0QB9Y3XX266KWYP3F grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=22 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:24.316" grpc.time_ms=22.601 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=1 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1957gitlab # [ 409.850497] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.557Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ce3c685e4dea49a52869e214","created_at":"2026-10-02T16:59:21.978Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.984Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ce3c685e4dea49a52869e214: start","job_status":"start","queue_duration_s":2.571991,"scheduling_latency_s":2.572021}1958gitlab # [ 409.867495] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.618Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["1"],"class":"PipelineHooksWorker","jid":"ce3c685e4dea49a52869e214","created_at":"2026-10-02T16:59:21.978Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7edae2f9320b4a04c72581c654fa0bced58f6822c07b16acc3b7bbc1333d0281","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.984Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-ce3c685e4dea49a52869e214: done: 0.061542 sec","job_status":"done","queue_duration_s":2.571991,"scheduling_latency_s":2.572021,"redis_calls":10,"redis_duration_s":0.003378,"redis_read_bytes":9,"redis_write_bytes":973,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000894,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":281,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.002051,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":506,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000433,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":186,"db_count":9,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":9,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":5,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.046528,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058702,"duration_s":0.061542,"completed_at":"2026-10-02T16:59:24.618Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.006913,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1959gitlab # [ 409.914827] gitaly[916]: time="2026-10-02T16:59:24.460Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:54.058" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.459" grpc.time_ms=0.817 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1960gitlab # [ 409.928675] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.621Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"22b3725b02b80e794d3f2ac2","created_at":"2026-10-02T16:59:21.993Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.997Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-22b3725b02b80e794d3f2ac2: start","job_status":"start","queue_duration_s":2.623346,"scheduling_latency_s":2.623374}1961gitlab # [ 409.946008] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.671Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"22b3725b02b80e794d3f2ac2","created_at":"2026-10-02T16:59:21.993Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:21.997Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-22b3725b02b80e794d3f2ac2: done: 0.050733 sec","job_status":"done","queue_duration_s":2.623346,"scheduling_latency_s":2.623374,"redis_calls":10,"redis_duration_s":0.009827,"redis_read_bytes":9,"redis_write_bytes":1023,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000625,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":295,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004543,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004659,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":198,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.025605,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00195968,"duration_s":0.050733,"completed_at":"2026-10-02T16:59:24.671Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.009707,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1962gitlab # [ 409.992111] gitaly[916]: time="2026-10-02T16:59:24.542Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=4 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=573 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=3 command.system_time_ms=4 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:34.013" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:24.534" grpc.time_ms=8.57 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1963gitlab # [ 410.006057] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.673Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8704b2cd8593ed1dde08b558","created_at":"2026-10-02T16:59:21.998Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.002Z","job_size_bytes":6,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8704b2cd8593ed1dde08b558: start","job_status":"start","queue_duration_s":2.670888,"scheduling_latency_s":2.670908}1964gitlab # [ 410.022442] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.716Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"8704b2cd8593ed1dde08b558","created_at":"2026-10-02T16:59:21.998Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.002Z","job_size_bytes":6,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-8704b2cd8593ed1dde08b558: done: 0.04282 sec","job_status":"done","queue_duration_s":2.670888,"scheduling_latency_s":2.670908,"redis_calls":10,"redis_duration_s":0.015311,"redis_read_bytes":9,"redis_write_bytes":1127,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003197,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":321,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.011039,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001075,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":224,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018014,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00433952,"duration_s":0.04282,"completed_at":"2026-10-02T16:59:24.716Z","load_balancing_strategy":"primary","db_duration_s":0.003036,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1965gitlab # [ 410.068116] gitaly[916]: time="2026-10-02T16:59:24.703Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWV1WAS576ND4T3A507F4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:54.502" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.702" grpc.time_ms=0.637 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1966gitlab # [ 410.082091] gitaly[916]: time="2026-10-02T16:59:24.862Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 diskcache=25d415c5-20d4-4b86-a88b-9a6737b7b242 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:59:24.853" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.853" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1967gitlab # [ 410.092384] nixbot[3482]: {"ts": "2026-10-02T16:59:24.999Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1968gitlab # [ 410.095281] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.718Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"7f4cddc2a56be2996e3fb0a8","created_at":"2026-10-02T16:59:22.004Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.009Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7f4cddc2a56be2996e3fb0a8: start","job_status":"start","queue_duration_s":2.709272,"scheduling_latency_s":2.709291}1969gitlab # [ 410.111425] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.760Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"7f4cddc2a56be2996e3fb0a8","created_at":"2026-10-02T16:59:22.004Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.009Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-7f4cddc2a56be2996e3fb0a8: done: 0.042342 sec","job_status":"done","queue_duration_s":2.709272,"scheduling_latency_s":2.709291,"redis_calls":10,"redis_duration_s":0.00225,"redis_read_bytes":9,"redis_write_bytes":989,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00035,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":285,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001452,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":514,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000448,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":190,"db_count":7,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029034,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00069752,"duration_s":0.042342,"completed_at":"2026-10-02T16:59:24.760Z","load_balancing_strategy":"primary","db_duration_s":0.006969,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1970gitlab # [ 410.156436] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:24 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3931971gitlab # [ 410.159030] gitaly[916]: time="2026-10-02T16:59:24.862Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=3 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=290 command.oublock=0 command.real_time_ms=4 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=CommitLanguages grpc.request.deadline="2026-10-02T22:59:24.853" grpc.request.fullMethod=/gitaly.CommitService/CommitLanguages grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=131 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=27 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.853" grpc.time_ms=8.643 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1972gitlab # [ 410.172785] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.762Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a5adc9d10e72c02b70679f8b","created_at":"2026-10-02T16:59:22.015Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.020Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a5adc9d10e72c02b70679f8b: start","job_status":"start","queue_duration_s":2.741357,"scheduling_latency_s":2.741375}1973gitlab # [ 410.189385] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.833Z","retry":25,"queue":"default","version":0,"store":null,"args":["1"],"class":"Ci::TrackFirstPipelineSucceededWorker","jid":"a5adc9d10e72c02b70679f8b","created_at":"2026-10-02T16:59:22.015Z","trace_propagation_headers":{"sentry-trace":"da2cba6e0c774c70ad7b001663ab2525-3c9ad178d79d4d89","baggage":"sentry-trace_id=da2cba6e0c774c70ad7b001663ab2525,sentry-release=Unknown"},"correlation_id":"01M3YRT41F6BRQJ0WGYB6DPGJ3","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c5d2c9faee55896683425d4b50feeb49f6074c2dda34904072dea18e4eda0a9b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.020Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackFirstPipelineSucceededWorker JID-a5adc9d10e72c02b70679f8b: done: 0.071616 sec","job_status":"done","queue_duration_s":2.741357,"scheduling_latency_s":2.741375,"redis_calls":14,"redis_duration_s":0.003569,"redis_read_bytes":14,"redis_write_bytes":1335,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":317,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001667,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":582,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001525,"redis_shared_state_read_bytes":8,"redis_shared_state_write_bytes":436,"db_count":12,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.011,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.048007,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.000322,"duration_s":0.071616,"completed_at":"2026-10-02T16:59:24.833Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013543,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1974gitlab # [ 410.235393] gitaly[916]: time="2026-10-02T16:59:24.876Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWV1WAS576ND4T3A507F4 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:34.247" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:24.867" grpc.time_ms=8.324 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1975gitlab # [ 410.249408] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.835Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0a8dfd4fd05d2f9c9f57876b","created_at":"2026-10-02T16:59:22.194Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b692c109d82ba28cce5138b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.198Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0a8dfd4fd05d2f9c9f57876b: start","job_status":"start","queue_duration_s":2.636708,"scheduling_latency_s":2.636726}1976gitlab # [ 410.267081] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.879Z","retry":1,"queue":"default","version":0,"store":null,"backtrace":5,"args":["1"],"class":"DetectRepositoryLanguagesWorker","jid":"0a8dfd4fd05d2f9c9f57876b","created_at":"2026-10-02T16:59:22.194Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:6013123ea9f6f8092f8eb4cb733d3949fa1e39a27e7ad74ad4f54a7c23d87776","duplicate-of":"b692c109d82ba28cce5138b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.198Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"DetectRepositoryLanguagesWorker JID-0a8dfd4fd05d2f9c9f57876b: done: 0.044407 sec","job_status":"done","queue_duration_s":2.636708,"scheduling_latency_s":2.636726,"gitaly_calls":1,"gitaly_duration_s":0.011259,"redis_calls":12,"redis_duration_s":0.002397,"redis_read_bytes":64,"redis_write_bytes":1136,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000948,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":479,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000465,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000984,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":520,"db_count":8,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.006,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.006,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026653,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.02608182,"duration_s":0.044407,"completed_at":"2026-10-02T16:59:24.879Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003839,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}1977gitlab # [ 410.315959] nixbot[3482]: {"ts": "2026-10-02T16:59:25.155Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 \"HTTP/1.1 201 Created\""}1978gitlab # [ 410.319393] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:25 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4fe91ef492212e25a90e9ab4b648d4a635f4e850 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 1491979gitlab # [ 410.322123] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 641980gitlab # [ 410.324449] gitaly[916]: time="2026-10-02T16:59:24.904Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:54.003" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.903" grpc.time_ms=0.691 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1981gitlab # [ 410.337375] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:24.881Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"beb077b160146b64ce527105","created_at":"2026-10-02T16:59:22.340Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:28a85005a8956187d08b7cd97c5f55b87376c7e8fd794e89c80d6c51a819723e","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.344Z","job_size_bytes":467,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-beb077b160146b64ce527105: start","job_status":"start","queue_duration_s":2.536901,"scheduling_latency_s":2.53692}1982gitlab # [ 410.355330] gitaly[916]: time="2026-10-02T16:59:24.915Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:59:34.006" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=142 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:24.907" grpc.time_ms=8.516 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1983gitlab # [ 410.368946] gitaly[916]: time="2026-10-02T16:59:24.951Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:59:54.049" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.950" grpc.time_ms=1.135 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1984gitlab # [ 410.382178] gitaly[916]: time="2026-10-02T16:59:24.957Z" level=info msg="finished streaming call with code NotFound" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 command.count=1 command.cpu_time_ms=1 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=37 command.oublock=0 command.real_time_ms=2 command.spawn_token_fork_ms=0 command.system_time_ms=1 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 error="tree entry not found" error_metadata="map[path:.gitlab-ci.yml]" grpc.code=NotFound grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=TreeEntry grpc.request.deadline="2026-10-02T16:59:54.054" grpc.request.fullMethod=/gitaly.CommitService/TreeEntry grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=186 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:24.955" grpc.time_ms=2.627 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1985gitlab # [ 410.399086] gitaly[916]: time="2026-10-02T16:59:24.993Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=2 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:34.084" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:24.984" grpc.time_ms=8.591 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1986gitlab # [ 410.412966] gitlab-workhorse[822]: time="2026-10-02T16:59:25Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=671987gitlab # [ 410.414872] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:25 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 701988gitlab # [ 410.416870] gitaly[916]: time="2026-10-02T16:59:25.059Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWVECM8F9N2673N712NKB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:55.058" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:25.058" grpc.time_ms=0.582 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1989gitlab # [ 410.429887] gitaly[916]: time="2026-10-02T16:59:25.103Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWVECM8F9N2673N712NKB grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:35.063" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:25.094" grpc.time_ms=9.444 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1990gitlab # [ 410.443979] gitaly[916]: time="2026-10-02T16:59:25.253Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=578 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=1 command.system_time_ms=0 command.user_time_ms=5 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWVM53YJG9SRHVP3R8XJG grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=188 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:25.242" grpc.time_ms=10.586 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1991gitlab # [ 410.457917] gitaly[916]: time="2026-10-02T16:59:25.325Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWVP7N5K9F13XNEGYQFT2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:25.314" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc1992gitlab # [ 410.468539] gitaly[916]: time="2026-10-02T16:59:25.325Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWVP7N5K9F13XNEGYQFT2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:25.314" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1993gitlab # [ 410.482192] gitaly[916]: time="2026-10-02T16:59:25.325Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=583 command.oublock=0 command.real_time_ms=6 command.spawn_token_fork_ms=1 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWVP7N5K9F13XNEGYQFT2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:25.314" grpc.time_ms=10.714 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc1994gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'1995gitlab # [ 410.640050] nixbot[3482]: INFO: ::ffff:127.0.0.1:53898 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK1996gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.05 seconds)1997gitlab # [ 410.764958] gitaly[916]: time="2026-10-02T16:59:25.701Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_info_count=1 catfile.read_info_ms=0 catfile.request_info_count=1 catfile.request_info_ms=0 command.count=3 command.cpu_time_ms=11 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=881 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=0 command.system_time_ms=7 command.user_time_ms=4 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=GetTreeEntries grpc.request.deadline="2026-10-02T16:59:55.084" grpc.request.fullMethod=/gitaly.CommitService/GetTreeEntries grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=178 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=103 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:25.685" grpc.time_ms=16.177 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1998gitlab # [ 410.789837] gitaly[916]: time="2026-10-02T16:59:25.719Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=574 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=4 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWKZGE3XDQYMGDTY205D5 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListFiles grpc.request.deadline="2026-10-02T16:59:55.104" grpc.request.fullMethod=/gitaly.CommitService/ListFiles grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=11 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:25.704" grpc.time_ms=14.763 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root1999gitlab # [ 410.810977] sidekiq-cluster[3364]: {"severity":"WARN","time":"2026-10-02T16:59:25.737Z","class":"CreatePipelineWorker","correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","project_id":1,"project_path":"root/test-flake","message":"Error creating pipeline","errors":"The pipeline did not run. Review the workflow:rules configuration for the pipeline.","pipeline_params":{"before":"4fe91ef492212e25a90e9ab4b648d4a635f4e850","after":"4a967074777ce7606287d14f5463040267a12c45","variables_attributes":[],"gitaly_context":{"glBuildId":"","scoped-user-id":""},"checkout_sha":"4a967074777ce7606287d14f5463040267a12c45","ref":"refs/heads/master"},"retry":0}2000gitlab # [ 410.821183] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.745Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_creation","args":["1","1","refs/heads/master","push","{\"inputs\"=\u003e{}}","[FILTERED]"],"class":"CreatePipelineWorker","jid":"beb077b160146b64ce527105","created_at":"2026-10-02T16:59:22.340Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"pipeline_composition","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:28a85005a8956187d08b7cd97c5f55b87376c7e8fd794e89c80d6c51a819723e","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.344Z","job_size_bytes":467,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"CreatePipelineWorker JID-beb077b160146b64ce527105: done: 0.86436 sec","job_status":"done","queue_duration_s":2.536901,"scheduling_latency_s":2.53692,"gitaly_calls":7,"gitaly_duration_s":0.068608,"redis_calls":33,"redis_duration_s":0.004621,"redis_read_bytes":186,"redis_write_bytes":1839,"redis_cache_calls":3,"redis_cache_duration_s":0.0005,"redis_cache_read_bytes":45,"redis_cache_write_bytes":255,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001041,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":443,"redis_repository_cache_calls":19,"redis_repository_cache_duration_s":0.001956,"redis_repository_cache_read_bytes":120,"redis_repository_cache_write_bytes":628,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001124,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":513,"db_count":27,"db_write_count":0,"db_cached_count":1,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.018,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":24,"db_ci_count":3,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.782958,"worker_id":"sidekiq_0","rate_limiting_gates":["pipelines_create","pipelines_created_per_user"],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.86436,"completed_at":"2026-10-02T16:59:25.745Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.01743,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2001gitlab # [ 410.874846] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.747Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{\"idempotency_key\"=\u003e\"79490721-90a6-4f00-a4ec-600f079d48a1\"}"],"class":"WebHookWorker","jid":"8841dea4b1992c04e7a04f5f","created_at":"2026-10-02T16:59:22.365Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c20fa50aab1e2a0b390a5aa8869354578cac6aa14eb52a02e661c2edf59188a1","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.369Z","job_size_bytes":1749,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8841dea4b1992c04e7a04f5f: start","job_status":"start","queue_duration_s":3.378589,"scheduling_latency_s":3.378624}2002gitlab # [ 410.893547] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.801Z","retry":4,"queue":"default","version":0,"store":null,"dead":false,"args":["1","[FILTERED]","push_hooks","{:idempotency_key=\u003e\"79490721-90a6-4f00-a4ec-600f079d48a1\"}"],"class":"WebHookWorker","jid":"8841dea4b1992c04e7a04f5f","created_at":"2026-10-02T16:59:22.365Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-fed9e52a7cdb4f5e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"Repositories::PostReceiveWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:c20fa50aab1e2a0b390a5aa8869354578cac6aa14eb52a02e661c2edf59188a1","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.369Z","job_size_bytes":1749,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHookWorker JID-8841dea4b1992c04e7a04f5f: done: 0.054232 sec","job_status":"done","queue_duration_s":3.378589,"scheduling_latency_s":3.378624,"redis_calls":14,"redis_duration_s":0.0033420000000000004,"redis_read_bytes":25,"redis_write_bytes":4684,"redis_queues_calls":2,"redis_queues_duration_s":0.000434,"redis_queues_read_bytes":2,"redis_queues_write_bytes":3728,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001744,"redis_queues_metadata_read_bytes":6,"redis_queues_metadata_write_bytes":630,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001164,"redis_shared_state_read_bytes":17,"redis_shared_state_write_bytes":326,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"external_http_count":1,"external_http_duration_s":0.007753760000014154,"cpu_s":0.036024,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.web_hook_worker.response_status":"success","extra.web_hook_worker.http_status":"ok","extra.web_hook_worker.signing_token_present":false,"duration_s":0.054232,"completed_at":"2026-10-02T16:59:25.801Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002799,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2003gitlab # [ 410.945242] nixbot[3482]: INFO: ::ffff:127.0.0.1:50662 - "POST /webhooks/gitlab HTTP/1.1" 202 Accepted2004gitlab # [ 410.947084] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.804Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"070a35edb70b796dfb2ac922","created_at":"2026-10-02T16:59:22.516Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-7574bcc8979340f2","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.524Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-070a35edb70b796dfb2ac922: start","job_status":"start","queue_duration_s":3.279616,"scheduling_latency_s":3.279646}2005gitlab # [ 410.963786] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.875Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::ScheduleAggregationWorker","jid":"070a35edb70b796dfb2ac922","created_at":"2026-10-02T16:59:22.516Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-7574bcc8979340f2","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"ProjectCacheWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:11bf4ab06a7a45f7b8fa79a254215eacd563749d4b07a335325f4d171a5a896a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:22.524Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ScheduleAggregationWorker JID-070a35edb70b796dfb2ac922: done: 0.07118 sec","job_status":"done","queue_duration_s":3.279616,"scheduling_latency_s":3.279646,"redis_calls":19,"redis_duration_s":0.0047079999999999995,"redis_read_bytes":21,"redis_write_bytes":4291,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000918,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":678,"redis_queues_calls":3,"redis_queues_duration_s":0.000664,"redis_queues_read_bytes":3,"redis_queues_write_bytes":2262,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.00208,"redis_queues_metadata_read_bytes":9,"redis_queues_metadata_write_bytes":1024,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.001046,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":327,"db_count":7,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":7,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.053196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.01163828,"duration_s":0.07118,"completed_at":"2026-10-02T16:59:25.875Z","load_balancing_strategy":"primary","db_duration_s":0.008598,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2006gitlab # [ 411.011674] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.877Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ccaae22078733d5de46cfa16"],"class":"WebHooks::LogExecutionWorker","jid":"2c2edfe9ea8c63c31689e14d","created_at":"2026-10-02T16:59:24.077Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-1d1ffda75f814f7d","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:485e910abdc1836f2366ff6be7364846470a5f445f5f708cf646498f64a17184","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:24.087Z","job_size_bytes":2593,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2c2edfe9ea8c63c31689e14d: start","job_status":"start","queue_duration_s":1.790166,"scheduling_latency_s":1.790194}2007gitlab # [ 411.029396] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.945Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","ccaae22078733d5de46cfa16"],"class":"WebHooks::LogExecutionWorker","jid":"2c2edfe9ea8c63c31689e14d","created_at":"2026-10-02T16:59:24.077Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-1d1ffda75f814f7d","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:485e910abdc1836f2366ff6be7364846470a5f445f5f708cf646498f64a17184","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:24.087Z","job_size_bytes":2593,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-2c2edfe9ea8c63c31689e14d: done: 0.067899 sec","job_status":"done","queue_duration_s":1.790166,"scheduling_latency_s":1.790194,"redis_calls":10,"redis_duration_s":0.005163,"redis_read_bytes":9,"redis_write_bytes":1045,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000571,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":299,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.004001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":542,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000591,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":204,"db_count":4,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.008,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.008,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.043496,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00058264,"duration_s":0.067899,"completed_at":"2026-10-02T16:59:25.945Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016794,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2008gitlab # [ 411.075858] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:25 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 832009gitlab # [ 411.078560] gitaly[916]: time="2026-10-02T16:59:25.897Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWW7Q1TNK5XWNCCPJ9YNM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=190 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:25.886" grpc.time_ms=10.483 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2010gitlab # [ 411.092416] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:25.947Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2cdb7eea1e2589ea626fa74f","created_at":"2026-10-02T16:59:24.575Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-2d0d10b3e67c497a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:24.579Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2cdb7eea1e2589ea626fa74f: start","job_status":"start","queue_duration_s":1.367825,"scheduling_latency_s":1.367855}2011gitlab # [ 411.147728] gitaly[916]: time="2026-10-02T16:59:26.084Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:26.083" grpc.time_ms=0.61 pid=916 span.kind=server system=grpc2012gitlab # [ 411.184880] gitaly[916]: time="2026-10-02T16:59:26.121Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:26.120" grpc.time_ms=0.691 pid=916 span.kind=server system=grpc2013gitlab # [ 411.197484] gitaly[916]: time="2026-10-02T16:59:26.125Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWAFT0TJ8B4P7N46ZF5R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.110" pid=916 remote_ip="2001:db8:1::1" request_sha=5d0ec876faced1ba584eb2e9d074c7a086d3e6eb response_bytes=67 span.kind=server system=grpc2014gitlab # [ 411.209411] gitlab-workhorse[822]: time="2026-10-02T16:59:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=672015gitlab # [ 411.211447] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 67 "" "git/2.55.0" 2232016gitlab # [ 411.213599] gitaly[916]: time="2026-10-02T16:59:26.126Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWAFT0TJ8B4P7N46ZF5R grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.110" negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2017gitlab # [ 411.228105] gitaly[916]: time="2026-10-02T16:59:26.126Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=9 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=584 command.oublock=0 command.real_time_ms=13 command.spawn_token_fork_ms=1 command.system_time_ms=9 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWAFT0TJ8B4P7N46ZF5R grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=6 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.110" grpc.time_ms=16.11 negotiation.command=ls-refs negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=0 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2018gitlab # [ 411.243309] gitaly[916]: time="2026-10-02T16:59:26.159Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:26.159" grpc.time_ms=0.462 pid=916 span.kind=server system=grpc2019gitlab # [ 411.253504] gitaly[916]: time="2026-10-02T16:59:26.165Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX diskcache=c8d7d7a9-961a-4eec-bd8a-5deeec54fafc grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:59:56.047" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:26.048" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2020gitlab # [ 411.263888] gitaly[916]: time="2026-10-02T16:59:26.166Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=98 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7403 command.oublock=0 command.real_time_ms=113 command.spawn_token_fork_ms=0 command.system_time_ms=70 command.user_time_ms=28 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T16:59:56.047" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:26.048" grpc.time_ms=117.935 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2021gitlab # [ 411.300302] gitaly[916]: time="2026-10-02T16:59:26.236Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:59:56.235" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=204 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:26.235" grpc.time_ms=1.01 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2022gitlab # [ 411.320985] gitaly[916]: time="2026-10-02T16:59:26.257Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=5 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=579 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=4 command.system_time_ms=5 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWK2SHH7MCCXBCYZWGGM grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=InfoRefsUploadPack grpc.request.fullMethod=/gitaly.SmartHTTPService/InfoRefsUploadPack grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=185 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.246" grpc.time_ms=10.464 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2023gitlab # [ 411.336883] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:26 +0000] "GET /root/test-flake.git/info/refs?service=git-upload-pack HTTP/1.1" 200 161 "" "git/2.55.0" 792024gitlab # [ 411.343157] gitaly[916]: time="2026-10-02T16:59:26.279Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWTVDE074YMAG3P551PPX grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:56.278" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:26.278" grpc.time_ms=0.909 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2025gitlab # [ 411.365768] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.302Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2cdb7eea1e2589ea626fa74f","created_at":"2026-10-02T16:59:24.575Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-2d0d10b3e67c497a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:24.579Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3364,"job_status":"rescheduled","message":"PipelineProcessWorker JID-2cdb7eea1e2589ea626fa74f: rescheduled"}2026gitlab # [ 411.387238] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.310Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"PipelineProcessWorker","jid":"2cdb7eea1e2589ea626fa74f","created_at":"2026-10-02T16:59:24.575Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-2d0d10b3e67c497a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:24.579Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-2cdb7eea1e2589ea626fa74f: done: 0.362813 sec","job_status":"done","queue_duration_s":1.367825,"scheduling_latency_s":1.367855,"gitaly_calls":3,"gitaly_duration_s":0.141343,"redis_calls":66,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.015739,"redis_read_bytes":148,"redis_write_bytes":18011,"redis_action_cable_calls":2,"redis_action_cable_duration_s":0.000693,"redis_action_cable_read_bytes":2,"redis_action_cable_write_bytes":407,"redis_cache_calls":9,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.00092,"redis_cache_read_bytes":17,"redis_cache_write_bytes":791,"redis_queues_calls":22,"redis_queues_duration_s":0.004081,"redis_queues_read_bytes":26,"redis_queues_write_bytes":12310,"redis_queues_metadata_calls":19,"redis_queues_metadata_duration_s":0.005137,"redis_queues_metadata_read_bytes":30,"redis_queues_metadata_write_bytes":3093,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000387,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.001683,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002145,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":30,"db_write_count":2,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":30,"db_replica_write_count":0,"db_primary_write_count":2,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.046,"db_main_txn_count":0,"db_ci_txn_count":1,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":16,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":2,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.01,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.01,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.027,"db_ci_duration_s":0.019,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.169018,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.31197036,"duration_s":0.362813,"completed_at":"2026-10-02T16:59:26.310Z","load_balancing_strategy":"primary","db_duration_s":0.046972,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2027gitlab # [ 411.449898] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.312Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8841dea4b1992c04e7a04f5f"],"class":"WebHooks::LogExecutionWorker","jid":"3681aca4d2adae1d644d326e","created_at":"2026-10-02T16:59:25.789Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-5902845a2950438e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:885a2c17c3daa8e8943010f3a0eaf69eadc54800e7a126b2fdfdcd9e5afd9a94","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:25.792Z","job_size_bytes":2591,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3681aca4d2adae1d644d326e: start","job_status":"start","queue_duration_s":0.519375,"scheduling_latency_s":0.519401}2028gitlab # [ 411.467857] gitaly[916]: time="2026-10-02T16:59:26.359Z" level=info msg="request details" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWNKJEEXG7KQN5WC7CM2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.340" pid=916 remote_ip="2001:db8:1::1" request_sha=22300437632a556c435888000544c7c9bb3428e9 response_bytes=216 span.kind=server system=grpc2029gitlab # [ 411.478622] gitlab-workhorse[822]: time="2026-10-02T16:59:26Z" level=info msg=git_traffic project_id=1 root_namespace_id=1 service=git-upload-pack written_bytes=2162030gitlab # [ 411.480589] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:26 +0000] "POST /root/test-flake.git/git-upload-pack HTTP/1.1" 200 216 "" "git/2.55.0" 1002031gitlab # [ 411.482565] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.340Z","retry":3,"queue":"default","version":0,"store":null,"args":["1","[FILTERED]","ok","8841dea4b1992c04e7a04f5f"],"class":"WebHooks::LogExecutionWorker","jid":"3681aca4d2adae1d644d326e","created_at":"2026-10-02T16:59:25.789Z","trace_propagation_headers":{"sentry-trace":"85243ebdfe0b4b31a98e7f4f4976bab4-5902845a2950438e","baggage":"sentry-trace_id=85243ebdfe0b4b31a98e7f4f4976bab4,sentry-release=Unknown"},"correlation_id":"01M3YRWKZGE3XDQYMGDTY205D5","meta.caller_id":"WebHookWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"webhooks","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","meta.related_class":"ProjectHook","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:885a2c17c3daa8e8943010f3a0eaf69eadc54800e7a126b2fdfdcd9e5afd9a94","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:25.792Z","job_size_bytes":2591,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"WebHooks::LogExecutionWorker JID-3681aca4d2adae1d644d326e: done: 0.028966 sec","job_status":"done","queue_duration_s":0.519375,"scheduling_latency_s":0.519401,"redis_calls":7,"redis_duration_s":0.001517,"redis_read_bytes":5,"redis_write_bytes":671,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001089,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000428,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":3,"db_write_count":1,"db_cached_count":0,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.004,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.004,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020397,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028966,"completed_at":"2026-10-02T16:59:26.340Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003724,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2032gitlab # [ 411.528934] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.342Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6b7cbbfd39c5f85ad67aaadf","created_at":"2026-10-02T16:59:25.854Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-a9e036971799457e","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:25.862Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6b7cbbfd39c5f85ad67aaadf: start","job_status":"start","queue_duration_s":0.47969,"scheduling_latency_s":0.479714}2033gitlab # [ 411.546516] gitaly[916]: time="2026-10-02T16:59:26.359Z" level=info msg="packfile negotiation complete" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWNKJEEXG7KQN5WC7CM2 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.340" negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2034gitlab # [ 411.560807] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.462Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"update_namespace_statistics","args":["1"],"class":"Namespaces::RootStatisticsWorker","jid":"6b7cbbfd39c5f85ad67aaadf","created_at":"2026-10-02T16:59:25.854Z","trace_propagation_headers":{"sentry-trace":"9641dc18c2eb4d7991eb9b0775b29e46-a9e036971799457e","baggage":"sentry-trace_id=9641dc18c2eb4d7991eb9b0775b29e46,sentry-release=Unknown"},"correlation_id":"01M3YRSZJ7E0R35JZ638W8XS7Q","meta.caller_id":"Namespaces::ScheduleAggregationWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"source_code_management","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.organization_id":1,"meta.root_caller_id":"POST /api/:version/internal/post_receive","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fdeb2b3453eb1185da615072e7fac8965876ce3585e2a5a18dfd6ce2ee6520d8","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:25.862Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::RootStatisticsWorker JID-6b7cbbfd39c5f85ad67aaadf: done: 0.119928 sec","job_status":"done","queue_duration_s":0.47969,"scheduling_latency_s":0.479714,"redis_calls":11,"redis_duration_s":0.002273,"redis_read_bytes":10,"redis_write_bytes":1189,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000377,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.001523,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":672,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000373,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":210,"db_count":12,"db_write_count":3,"db_cached_count":0,"db_txn_count":3,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.019,"db_main_txn_count":3,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":12,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":3,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.005,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.009,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.019,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.081502,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00025286,"duration_s":0.119928,"completed_at":"2026-10-02T16:59:26.462Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.028352,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2035gitlab # [ 411.607622] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.464Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5e4be3e43059684120f74453","created_at":"2026-10-02T16:59:26.036Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.039Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5e4be3e43059684120f74453: start","job_status":"start","queue_duration_s":0.424622,"scheduling_latency_s":0.424644}2036gitlab # [ 411.624888] gitaly[916]: time="2026-10-02T16:59:26.360Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=14 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=1103 command.oublock=0 command.real_time_ms=16 command.spawn_token_fork_ms=1 command.system_time_ms=14 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWNKJEEXG7KQN5WC7CM2 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-workhorse grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=PostUploadPackWithSidechannel grpc.request.fullMethod=/gitaly.SmartHTTPService/PostUploadPackWithSidechannel grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=134 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=20 grpc.service=gitaly.SmartHTTPService grpc.start_time="2026-10-02T16:59:26.340" grpc.time_ms=19.358 negotiation.command=fetch negotiation.filter="blob:none" negotiation.haves=0 negotiation.user_agent=git/2.55.0-Linux negotiation.wants=1 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc2037gitlab: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45'2038gitlab # [ 411.640400] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.516Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"5e4be3e43059684120f74453","created_at":"2026-10-02T16:59:26.036Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f0b0c0ccc9472d6819984cd49b3aa8829dfa124660b99391ec24c08cadec0f09","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.039Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-5e4be3e43059684120f74453: done: 0.051884 sec","job_status":"done","queue_duration_s":0.424622,"scheduling_latency_s":0.424644,"redis_calls":7,"redis_duration_s":0.0016849999999999999,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001264,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000421,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.034504,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.051884,"completed_at":"2026-10-02T16:59:26.516Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.013602,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2039gitlab: (finished: must succeed: curl --fail -s 'http://127.0.0.1:8010/api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45', in 0.06 seconds)2040(finished: subtest: push delivers the webhook, in 92.51 seconds)2041subtest: eval and build post commit statuses2042gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2043gitlab # [ 411.684972] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.517Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d0d05aa5df3afce49eb3c83a","created_at":"2026-10-02T16:59:26.173Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.178Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d0d05aa5df3afce49eb3c83a: start","job_status":"start","queue_duration_s":0.339715,"scheduling_latency_s":0.339735}2044gitlab # [ 411.702391] gitaly[916]: time="2026-10-02T16:59:26.480Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:56.077" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:26.478" grpc.time_ms=0.964 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2045gitlab # [ 411.716824] nixbot[3482]: INFO: ::ffff:127.0.0.1:50670 - "GET /api/repos/gitlab/root/test-flake/builds?commit=4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 200 OK2046gitlab # [ 411.718942] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.551Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"d0d05aa5df3afce49eb3c83a","created_at":"2026-10-02T16:59:26.173Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:516356bdbc9d7b42e82049f84b2c5e8c1a6e759dfce68fef2843f4d2de5c13d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.178Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-d0d05aa5df3afce49eb3c83a: done: 0.033996 sec","job_status":"done","queue_duration_s":0.339715,"scheduling_latency_s":0.339735,"redis_calls":7,"redis_duration_s":0.0021060000000000002,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001596,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00051,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022572,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.033996,"completed_at":"2026-10-02T16:59:26.551Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005593,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2047gitlab # [ 411.765553] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.553Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"118197e13a643ac22763f8f7","created_at":"2026-10-02T16:59:26.189Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.192Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-118197e13a643ac22763f8f7: start","job_status":"start","queue_duration_s":0.360648,"scheduling_latency_s":0.360672}2048gitlab # [ 411.782175] gitaly[916]: time="2026-10-02T16:59:26.507Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=589 command.oublock=0 command.real_time_ms=10 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListBranchNamesContainingCommit grpc.request.deadline="2026-10-02T16:59:56.094" grpc.request.fullMethod=/gitaly.RefService/ListBranchNamesContainingCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=8 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:26.494" grpc.time_ms=12.059 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2049gitlab # [ 411.796656] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.589Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"118197e13a643ac22763f8f7","created_at":"2026-10-02T16:59:26.189Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:63f9f7cd25437cd73f59d7b8712965562d29d563b949d0eb0d88506f86c16e46","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.192Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-118197e13a643ac22763f8f7: done: 0.036597 sec","job_status":"done","queue_duration_s":0.360648,"scheduling_latency_s":0.360672,"redis_calls":7,"redis_duration_s":0.001309,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000964,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000345,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.024396,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036597,"completed_at":"2026-10-02T16:59:26.589Z","load_balancing_strategy":"primary","db_duration_s":0.007212,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2050gitlab # [ 411.840475] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.591Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e6cbb238419fd31c17bb9bdf","created_at":"2026-10-02T16:59:26.203Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.205Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e6cbb238419fd31c17bb9bdf: start","job_status":"start","queue_duration_s":0.385985,"scheduling_latency_s":0.386009}2051gitlab # [ 411.856416] gitaly[916]: time="2026-10-02T16:59:26.524Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=7 command.spawn_token_fork_ms=0 command.system_time_ms=3 command.user_time_ms=3 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:36.005" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:26.515" grpc.time_ms=9.011 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2052gitlab # [ 411.870385] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.638Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::TestFailureHistoryWorker","jid":"e6cbb238419fd31c17bb9bdf","created_at":"2026-10-02T16:59:26.203Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:d13d7384904dcf1f669f77a046ff89ec7da9f34ad97cdf916c152a957ed611c0","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.205Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-e6cbb238419fd31c17bb9bdf: done: 0.047245 sec","job_status":"done","queue_duration_s":0.385985,"scheduling_latency_s":0.386009,"redis_calls":10,"redis_duration_s":0.0037990000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000906,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.002556,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000337,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.009,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026858,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.047245,"completed_at":"2026-10-02T16:59:26.638Z","load_balancing_strategy":"primary","db_duration_s":0.013782,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2053gitlab # [ 411.915691] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.640Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"27a5bee1a62ea8d5a3ba9bad","created_at":"2026-10-02T16:59:26.205Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.208Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-27a5bee1a62ea8d5a3ba9bad: start","job_status":"start","queue_duration_s":0.431936,"scheduling_latency_s":0.431965}2054gitlab # [ 411.934712] gitaly[916]: time="2026-10-02T16:59:26.552Z" level=info msg="finished unary call with code OK" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=ObjectFormat grpc.request.deadline="2026-10-02T16:59:36.041" grpc.request.fullMethod=/gitaly.RepositoryService/ObjectFormat grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=123 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:26.551" grpc.time_ms=0.392 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2055gitlab # [ 411.946800] nixbot[3482]: {"ts": "2026-10-02T16:59:26.860Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 \"HTTP/1.1 201 Created\""}2056gitlab # [ 411.949766] nixbot[3482]: {"ts": "2026-10-02T16:59:26.874Z", "level": "info", "logger": "nixbot.flake_prefetch", "msg": "prefetching flake inputs", "flake_dir": "/var/lib/nixbot/worktrees/1-4a967074777c-5"}2057gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.30 seconds)2058gitlab # [ 411.952543] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.676Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["1"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"27a5bee1a62ea8d5a3ba9bad","created_at":"2026-10-02T16:59:26.205Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:dec604319992be0929da6e2532a6b9409f09b002c6422a6643ce62aa4e5896b9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.208Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-27a5bee1a62ea8d5a3ba9bad: done: 0.036234 sec","job_status":"done","queue_duration_s":0.431936,"scheduling_latency_s":0.431965,"redis_calls":7,"redis_duration_s":0.001521,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001147,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000374,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.023185,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036234,"completed_at":"2026-10-02T16:59:26.676Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00761,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2059gitlab # [ 411.997530] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.678Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ed11f9c05fb1a1810ee952fa","created_at":"2026-10-02T16:59:26.209Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.211Z","job_size_bytes":19,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ed11f9c05fb1a1810ee952fa: start","job_status":"start","queue_duration_s":0.466442,"scheduling_latency_s":0.46647}2060gitlab # [ 412.013917] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:26 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 201 637 "" "python-httpx/0.28.1" 4622061gitlab # [ 412.016557] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:26 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 2422062gitlab # [ 412.019223] gitaly[916]: time="2026-10-02T16:59:26.569Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=6 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=1 command.system_time_ms=6 command.user_time_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=RefExists grpc.request.deadline="2026-10-02T16:59:36.049" grpc.request.fullMethod=/gitaly.RefService/RefExists grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=182 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:26.560" grpc.time_ms=9.487 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2063gitlab # [ 412.033355] gitaly[916]: time="2026-10-02T16:59:26.628Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4152 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:26.624" grpc.time_ms=0.612 pid=916 span.kind=server system=grpc2064gitlab # [ 412.043429] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.706Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"ed11f9c05fb1a1810ee952fa","created_at":"2026-10-02T16:59:26.209Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_delivery","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:92e420870d25c52b5ebdf32286e1720fa4ac32c28b0440972437d5366aa8550b","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.211Z","job_size_bytes":19,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"AutoMergeProcessWorker JID-ed11f9c05fb1a1810ee952fa: done: 0.028064 sec","job_status":"done","queue_duration_s":0.466442,"scheduling_latency_s":0.46647,"redis_calls":8,"redis_duration_s":0.001205,"redis_read_bytes":6,"redis_write_bytes":764,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.000914,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":570,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000291,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":194,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018126,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028064,"completed_at":"2026-10-02T16:59:26.706Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004044,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2065gitlab # [ 412.087857] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.707Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b3e96e86acdc4fcb66b7e3e8","created_at":"2026-10-02T16:59:26.213Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.215Z","job_size_bytes":66,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b3e96e86acdc4fcb66b7e3e8: start","job_status":"start","queue_duration_s":0.492176,"scheduling_latency_s":0.492197}2066gitlab # [ 412.106422] gitaly[916]: time="2026-10-02T16:59:26.654Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=4154 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:26.653" grpc.time_ms=0.542 pid=916 span.kind=server system=grpc2067gitlab # [ 412.116514] gitaly[916]: time="2026-10-02T16:59:26.665Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA diskcache=932866c4-9c7c-4d3d-9b88-d8ba5bae327f grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:59:36.061" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:26.572" pid=916 ref=refs/keep-around/4a967074777ce7606287d14f5463040267a12c45 remote_ip="2001:db8:1::1" revision=4a967074777ce7606287d14f5463040267a12c45 span.kind=server system=grpc username=root2068gitlab # [ 412.128258] nixbot[3482]: {"ts": "2026-10-02T16:59:27.005Z", "level": "info", "logger": "nixbot.nix_eval", "msg": "starting evaluation", "worktree": "/var/lib/nixbot/worktrees/1-4a967074777c-5", "argv0": "bwrap"}2069gitlab # [ 412.130727] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.732Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineFinishedEvent","{\"pipeline_id\"=\u003e1, \"status\"=\u003e\"success\"}"],"class":"Ci::Workloads::UpdateWorkloadStatusEventWorker","jid":"b3e96e86acdc4fcb66b7e3e8","created_at":"2026-10-02T16:59:26.213Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:f786010d22acb06782f1370d01e07bde28bed0498922adde5f6769effe4cc7f7","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.215Z","job_size_bytes":66,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Workloads::UpdateWorkloadStatusEventWorker JID-b3e96e86acdc4fcb66b7e3e8: done: 0.025108 sec","job_status":"done","queue_duration_s":0.492176,"scheduling_latency_s":0.492197,"redis_calls":7,"redis_duration_s":0.001178,"redis_read_bytes":5,"redis_write_bytes":761,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000835,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000343,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":240,"db_count":5,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":5,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01691,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025108,"completed_at":"2026-10-02T16:59:26.732Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003526,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2070gitlab # [ 412.177848] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.734Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ba2b50dcdcb9e49805c82557","created_at":"2026-10-02T16:59:26.216Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.218Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ba2b50dcdcb9e49805c82557: start","job_status":"start","queue_duration_s":0.515815,"scheduling_latency_s":0.515836}2071gitlab # [ 412.194944] gitaly[916]: time="2026-10-02T16:59:26.665Z" level=info msg="finished unary call with code OK" command.count=2 command.cpu_time_ms=75 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=5060 command.oublock=8 command.real_time_ms=83 command.spawn_token_fork_ms=0 command.system_time_ms=24 command.user_time_ms=51 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=WriteRef grpc.request.deadline="2026-10-02T16:59:36.061" grpc.request.fullMethod=/gitaly.RepositoryService/WriteRef grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=224 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RepositoryService grpc.start_time="2026-10-02T16:59:26.572" grpc.time_ms=93.072 pid=916 ref=refs/keep-around/4a967074777ce7606287d14f5463040267a12c45 remote_ip="2001:db8:1::1" revision=4a967074777ce7606287d14f5463040267a12c45 span.kind=server system=grpc username=root2072gitlab # [ 412.210299] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.770Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["1"],"class":"Ci::PipelineFinishedWorker","jid":"ba2b50dcdcb9e49805c82557","created_at":"2026-10-02T16:59:26.216Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:5c8b4490d8f5ea379beda67cd2fb00a658e7ee64e701adca381fbf2e7fc7314f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.218Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineFinishedWorker JID-ba2b50dcdcb9e49805c82557: done: 0.036195 sec","job_status":"done","queue_duration_s":0.515815,"scheduling_latency_s":0.515836,"redis_calls":7,"redis_duration_s":0.002458,"redis_read_bytes":5,"redis_write_bytes":656,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.002109,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":458,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":198,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.012,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":1,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.011,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018794,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036195,"completed_at":"2026-10-02T16:59:26.770Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.0121,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2073gitlab # [ 412.255151] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.779Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1ed2f2756d6fa102a34628e9","created_at":"2026-10-02T16:59:26.218Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.220Z","job_size_bytes":6,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1ed2f2756d6fa102a34628e9: start","job_status":"start","queue_duration_s":0.558573,"scheduling_latency_s":0.558602}2074gitlab # [ 412.271935] gitaly[916]: time="2026-10-02T16:59:26.675Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:56.274" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:26.674" grpc.time_ms=0.438 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2075gitlab # [ 412.285407] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.804Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["[FILTERED]"],"class":"Ci::Refs::UnlockPreviousPipelinesWorker","jid":"1ed2f2756d6fa102a34628e9","created_at":"2026-10-02T16:59:26.218Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a0f041753de3870d67154f15c09ece63e0c4ee06ec0f7cfd2b7747953ad06d9d","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.220Z","job_size_bytes":6,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Refs::UnlockPreviousPipelinesWorker JID-1ed2f2756d6fa102a34628e9: done: 0.024821 sec","job_status":"done","queue_duration_s":0.558573,"scheduling_latency_s":0.558602,"redis_calls":7,"redis_duration_s":0.005158,"redis_read_bytes":5,"redis_write_bytes":721,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004659,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":497,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000499,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":224,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":1,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.001,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.0123,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.024821,"completed_at":"2026-10-02T16:59:26.804Z","load_balancing_strategy":"primary","db_duration_s":0.001317,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2076gitlab # [ 412.329258] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.809Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bfb39a69e7de381d3453c6fb","created_at":"2026-10-02T16:59:26.221Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.223Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bfb39a69e7de381d3453c6fb: start","job_status":"start","queue_duration_s":0.586385,"scheduling_latency_s":0.586405}2077gitlab # [ 412.346287] gitaly[916]: time="2026-10-02T16:59:26.860Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWX27MTWZC2VK2F4GGXGD grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:56.458" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:26.859" grpc.time_ms=0.725 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2078gitlab # [ 412.360094] nix-daemon[3641]: accepted connection from pid 4731, user nixbot2079gitlab # [ 412.361115] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.841Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["1"],"class":"PipelineMetricsWorker","jid":"bfb39a69e7de381d3453c6fb","created_at":"2026-10-02T16:59:26.221Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:644fde72c31829615c47ecd7736206eb57ccdd526788a2927a77070bca65b393","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.223Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineMetricsWorker JID-bfb39a69e7de381d3453c6fb: done: 0.031999 sec","job_status":"done","queue_duration_s":0.586385,"scheduling_latency_s":0.586405,"redis_calls":7,"redis_duration_s":0.006903,"redis_read_bytes":5,"redis_write_bytes":636,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.005006,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":446,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.001897,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.018566,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.031999,"completed_at":"2026-10-02T16:59:26.841Z","load_balancing_strategy":"primary","db_duration_s":0.003872,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2080gitlab # [ 412.404105] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.845Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ddb6833d8a422da128a869f2","created_at":"2026-10-02T16:59:26.302Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.304Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ddb6833d8a422da128a869f2: start","job_status":"start","queue_duration_s":0.540399,"scheduling_latency_s":0.54042}2081gitlab # [ 412.420079] gitaly[916]: time="2026-10-02T16:59:27.089Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T16:59:57.087" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:27.087" grpc.time_ms=1.287 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2082gitlab # [ 412.433232] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.880Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","rescheduled_once":true,"args":["1"],"class":"PipelineProcessWorker","jid":"ddb6833d8a422da128a869f2","created_at":"2026-10-02T16:59:26.302Z","trace_propagation_headers":{"sentry-trace":"39241d7eb59f4658bc9177cc00f242aa-97793ec5ea3a4a0a","baggage":"sentry-trace_id=39241d7eb59f4658bc9177cc00f242aa,sentry-release=Unknown"},"correlation_id":"01M3YRWTVDE074YMAG3P551PPX","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:f17dab72d818b5591168c35c7bdfefd02fa3f68167592daadd7c40e5b811ce78","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.304Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ddb6833d8a422da128a869f2: done: 0.035566 sec","job_status":"done","queue_duration_s":0.540399,"scheduling_latency_s":0.54042,"redis_calls":8,"redis_duration_s":0.004704,"redis_read_bytes":6,"redis_write_bytes":754,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.001456,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":564,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003248,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":190,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02264,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035566,"completed_at":"2026-10-02T16:59:26.880Z","load_balancing_strategy":"primary","db_duration_s":0.005975,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2083gitlab # [ 412.476471] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.887Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"780a71bd4119a3e51c247283","created_at":"2026-10-02T16:59:26.704Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.711Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-780a71bd4119a3e51c247283: start","job_status":"start","queue_duration_s":0.176219,"scheduling_latency_s":0.176245}2084gitlab # [ 412.493857] gitaly[916]: time="2026-10-02T16:59:27.130Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWWSWANQVJ1QF5DMAA6QA grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:57.129" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:27.129" grpc.time_ms=0.935 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2085gitlab # [ 412.506675] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.923Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"MergeRequests::UpdateHeadPipelineWorker","jid":"780a71bd4119a3e51c247283","created_at":"2026-10-02T16:59:26.704Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_review_workflow","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:87cea9682199b75c272b7ff1dd93d7c9f14b347650bc2388e7fda305bf1df003","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.711Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::UpdateHeadPipelineWorker JID-780a71bd4119a3e51c247283: done: 0.036544 sec","job_status":"done","queue_duration_s":0.176219,"scheduling_latency_s":0.176245,"redis_calls":7,"redis_duration_s":0.001446,"redis_read_bytes":5,"redis_write_bytes":731,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001079,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":503,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000367,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":228,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.005,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.002,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026598,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.036544,"completed_at":"2026-10-02T16:59:26.923Z","load_balancing_strategy":"primary","db_duration_s":0.004855,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2086gitlab # [ 412.552455] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.925Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ccc6e87541d8fb7d0de4a22e","created_at":"2026-10-02T16:59:26.712Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.723Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ccc6e87541d8fb7d0de4a22e: start","job_status":"start","queue_duration_s":0.201606,"scheduling_latency_s":0.20163}2087gitlab # [ 412.570485] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.995Z","retry":25,"queue":"default","version":0,"store":null,"args":["Ci::PipelineCreatedEvent","{\"pipeline_id\"=\u003e2, \"partition_id\"=\u003e100}"],"class":"Ci::TrackPipelineTriggerEventsWorker","jid":"ccc6e87541d8fb7d0de4a22e","created_at":"2026-10-02T16:59:26.712Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:fd81ef6159ba89c96a4a693456f0ec7f47423572cce07ee544ec449d0efb8f9c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.723Z","job_size_bytes":65,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TrackPipelineTriggerEventsWorker JID-ccc6e87541d8fb7d0de4a22e: done: 0.07051 sec","job_status":"done","queue_duration_s":0.201606,"scheduling_latency_s":0.20163,"redis_calls":10,"redis_duration_s":0.001731,"redis_read_bytes":14,"redis_write_bytes":857,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000862,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":494,"redis_shared_state_calls":5,"redis_shared_state_duration_s":0.000869,"redis_shared_state_read_bytes":10,"redis_shared_state_write_bytes":363,"db_count":11,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":11,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.014,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.011,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.050793,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.07051,"completed_at":"2026-10-02T16:59:26.995Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.014895,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2088gitlab # [ 412.615860] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:26.997Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ef128795845416cdb69a12de","created_at":"2026-10-02T16:59:26.823Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.826Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ef128795845416cdb69a12de: start","job_status":"start","queue_duration_s":0.171312,"scheduling_latency_s":0.171342}2089gitlab # [ 412.632486] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:27.148Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"ef128795845416cdb69a12de","created_at":"2026-10-02T16:59:26.823Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-e9589714823748fa","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:26.826Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-ef128795845416cdb69a12de: done: 0.151362 sec","job_status":"done","queue_duration_s":0.171312,"scheduling_latency_s":0.171342,"gitaly_calls":2,"gitaly_duration_s":0.007517,"redis_calls":34,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.00787,"redis_read_bytes":79,"redis_write_bytes":5121,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.000898,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":10,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001088,"redis_cache_read_bytes":19,"redis_cache_write_bytes":892,"redis_queues_calls":2,"redis_queues_duration_s":0.000259,"redis_queues_read_bytes":2,"redis_queues_write_bytes":1132,"redis_queues_metadata_calls":7,"redis_queues_metadata_duration_s":0.002933,"redis_queues_metadata_read_bytes":7,"redis_queues_metadata_write_bytes":772,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000238,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":2,"redis_repository_cache_duration_s":0.000461,"redis_repository_cache_read_bytes":39,"redis_repository_cache_write_bytes":83,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.001095,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":551,"db_count":27,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":27,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.028,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":14,"db_ci_count":13,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.003,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.005,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.015,"db_ci_duration_s":0.013,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.10937,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.10353338,"duration_s":0.151362,"completed_at":"2026-10-02T16:59:27.148Z","load_balancing_strategy":"primary","db_duration_s":0.029772,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2090gitlab # [ 412.685919] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:27.151Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2fe062b27c7cdf90150be01a","created_at":"2026-10-02T16:59:27.069Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-97db3cce6fe14284","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:27.071Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2fe062b27c7cdf90150be01a: start","job_status":"start","queue_duration_s":0.079555,"scheduling_latency_s":0.07959}2091gitlab # [ 412.702843] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:27.200Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_hooks","args":["2"],"class":"PipelineHooksWorker","jid":"2fe062b27c7cdf90150be01a","created_at":"2026-10-02T16:59:27.069Z","trace_propagation_headers":{"sentry-trace":"e6a3d5067c1a43c99a4cbd7b81758753-97db3cce6fe14284","baggage":"sentry-trace_id=e6a3d5067c1a43c99a4cbd7b81758753,sentry-release=Unknown"},"correlation_id":"01M3YRWWSWANQVJ1QF5DMAA6QA","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4f872504460a03cbbb1f784e98442edd2efbf9bca51705c4112dc946be2ae20e","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:27.071Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineHooksWorker JID-2fe062b27c7cdf90150be01a: done: 0.049428 sec","job_status":"done","queue_duration_s":0.079555,"scheduling_latency_s":0.07959,"redis_calls":7,"redis_duration_s":0.0017200000000000002,"redis_read_bytes":5,"redis_write_bytes":626,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001278,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":440,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000442,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":186,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.004,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.036925,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.049428,"completed_at":"2026-10-02T16:59:27.200Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00699,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2092gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2093gitlab # [ 413.125097] gitaly[916]: time="2026-10-02T16:59:28.061Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWYBG4WJD0YD06Q4S1JJT grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:58.059" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:28.060" grpc.time_ms=0.857 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2094gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.17 seconds)2095gitlab # [ 413.167287] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:28 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1182096gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2097gitlab # [ 414.268222] gitaly[916]: time="2026-10-02T16:59:29.204Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRWZFHZW88SNJTSWT0F8SJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T16:59:59.202" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:29.202" grpc.time_ms=0.941 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2098gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.13 seconds)2099gitlab # [ 414.302253] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:29 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1002100gitlab # [ 414.655416] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.590Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1d6df625c3204d14c65d9158","created_at":"2026-10-02T16:59:29.567Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"665a58e6eff55142180a9392e18275ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.587Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1d6df625c3204d14c65d9158: start","job_status":"start","queue_duration_s":0.00309,"scheduling_latency_s":0.003127}2101gitlab # [ 414.775907] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.712Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"MergeRequests::ProcessScheduledMergeWorker","jid":"1d6df625c3204d14c65d9158","created_at":"2026-10-02T16:59:29.567Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"665a58e6eff55142180a9392e18275ab","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a6d655d277495cf53e182e80822575dacc8a3389eea573549908a88faf373827","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.587Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequests::ProcessScheduledMergeWorker JID-1d6df625c3204d14c65d9158: done: 0.121223 sec","job_status":"done","queue_duration_s":0.00309,"scheduling_latency_s":0.003127,"redis_calls":11,"redis_duration_s":0.012669,"redis_read_bytes":9,"redis_write_bytes":1399,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002583,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006779,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.003307,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":560,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032463,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00257534,"exclusive_lock_hold_duration_s":0.01242476,"duration_s":0.121223,"completed_at":"2026-10-02T16:59:29.711Z","load_balancing_strategy":"primary","db_duration_s":0.012976,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2102gitlab # [ 414.841361] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.715Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a14c21f47744a07c47c6cd44","created_at":"2026-10-02T16:59:29.632Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8bc9b7cecdcd0f7a3c2eecca8e6c438","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.641Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a14c21f47744a07c47c6cd44: start","job_status":"start","queue_duration_s":0.074598,"scheduling_latency_s":0.074628}2103gitlab # [ 414.927323] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.862Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"BatchedGitRefUpdates::CleanupSchedulerWorker","jid":"a14c21f47744a07c47c6cd44","created_at":"2026-10-02T16:59:29.632Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"b8bc9b7cecdcd0f7a3c2eecca8e6c438","meta.root_caller_id":"Cronjob","meta.feature_category":"gitaly","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:1df286e3cb94a9fae913bbc935abcb74e2ce04a919d8b126d0730a03d3a1570c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.641Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"BatchedGitRefUpdates::CleanupSchedulerWorker JID-a14c21f47744a07c47c6cd44: done: 0.146926 sec","job_status":"done","queue_duration_s":0.074598,"scheduling_latency_s":0.074628,"redis_calls":11,"redis_duration_s":0.013267000000000001,"redis_read_bytes":9,"redis_write_bytes":1418,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.003327,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":331,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":521,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.005051,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":566,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.008,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.008,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.030578,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00125446,"exclusive_lock_hold_duration_s":0.00575104,"extra.batched_git_ref_updates_cleanup_scheduler_worker.stats":{"total_projects":0},"duration_s":0.146926,"completed_at":"2026-10-02T16:59:29.862Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.007755,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2104gitlab # [ 414.993539] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.871Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"facb0e60df3d50a22bd54910","created_at":"2026-10-02T16:59:29.644Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34583b6b3bf242189656eca03a7e4e6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.662Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-facb0e60df3d50a22bd54910: start","job_status":"start","queue_duration_s":0.209481,"scheduling_latency_s":0.209513}2105gitlab # [ 415.033703] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.969Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerCellLocalWorker","jid":"facb0e60df3d50a22bd54910","created_at":"2026-10-02T16:59:29.644Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"34583b6b3bf242189656eca03a7e4e6c","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4ae89c58d1762b2d31a89d6fdeef88f3d5e03409467fbec711d482dc740c1b84","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.662Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerCellLocalWorker JID-facb0e60df3d50a22bd54910: done: 0.097905 sec","job_status":"done","queue_duration_s":0.209481,"scheduling_latency_s":0.209513,"redis_calls":11,"redis_duration_s":0.014722,"redis_read_bytes":9,"redis_write_bytes":1459,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00366,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":628,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.007472,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":563,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.00359,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":268,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.007,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.029768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.097905,"completed_at":"2026-10-02T16:59:29.969Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.00718,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2106gitlab # [ 415.099834] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:29.977Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"79c8f48cbf1756286a6bf20d","created_at":"2026-10-02T16:59:29.679Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d02cae1f3f21c4b56098e5b58d5a184","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.720Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-79c8f48cbf1756286a6bf20d: start","job_status":"start","queue_duration_s":0.25682,"scheduling_latency_s":0.256854}2107gitlab # [ 415.118910] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.032Z","class":"Database::BackgroundOperation::SecSchedulerWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerWorker execution for unconfigured database sec","retry":0}2108gitlab # [ 415.123170] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.053Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerWorker","jid":"79c8f48cbf1756286a6bf20d","created_at":"2026-10-02T16:59:29.679Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"8d02cae1f3f21c4b56098e5b58d5a184","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:7bbb9537dbfe97e82719c1aa12366766ebff3829230f03761baf75912b8c8947","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.720Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerWorker JID-79c8f48cbf1756286a6bf20d: done: 0.076192 sec","job_status":"done","queue_duration_s":0.25682,"scheduling_latency_s":0.256854,"redis_calls":9,"redis_duration_s":0.01027,"redis_read_bytes":7,"redis_write_bytes":1112,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00149,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":341,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.006679,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.002101,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.019457,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.076192,"completed_at":"2026-10-02T16:59:30.052Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.002776,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2109gitlab # [ 415.169280] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.063Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"55099f90921159d53012c2a3","created_at":"2026-10-02T16:59:29.780Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"779d22e3783dbc85ba3a98804ba13d7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.794Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-55099f90921159d53012c2a3: start","job_status":"start","queue_duration_s":0.269244,"scheduling_latency_s":0.269277}2110gitlab # [ 415.211542] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.147Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::MainSchedulerWorker","jid":"55099f90921159d53012c2a3","created_at":"2026-10-02T16:59:29.780Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"779d22e3783dbc85ba3a98804ba13d7a","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:82a961bb8ace1086727bc80b4cde448c10530a6fb8b4c336ffe5d1d9d343271f","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.794Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::MainSchedulerWorker JID-55099f90921159d53012c2a3: done: 0.083846 sec","job_status":"done","queue_duration_s":0.269244,"scheduling_latency_s":0.269277,"redis_calls":9,"redis_duration_s":0.011365,"redis_read_bytes":7,"redis_write_bytes":1119,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002614,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":343,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00502,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":530,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.003731,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":246,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.020778,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.083846,"completed_at":"2026-10-02T16:59:30.147Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005508,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2111gitlab # [ 415.272327] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.154Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bbea76fdc97a6b8f611168e3","created_at":"2026-10-02T16:59:29.806Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6244f35f20ca7551ceddc7e1aed1973","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.828Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bbea76fdc97a6b8f611168e3: start","job_status":"start","queue_duration_s":0.326123,"scheduling_latency_s":0.326147}2112gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2113gitlab # [ 415.410159] gitaly[916]: time="2026-10-02T16:59:30.346Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX0K5BQE6XS5X1SCBV1N0 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:00.344" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:30.344" grpc.time_ms=0.822 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2114gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.14 seconds)2115gitlab # [ 415.442629] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:30 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1002116gitlab # [ 415.748088] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.683Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ConcurrencyLimit::ResumeWorker","jid":"bbea76fdc97a6b8f611168e3","created_at":"2026-10-02T16:59:29.806Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"e6244f35f20ca7551ceddc7e1aed1973","meta.root_caller_id":"Cronjob","meta.feature_category":"scalability","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bc8a9466f77693e94cb5ebf1ae9ff94995b6779d8794f59890276364c8a3c78a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.828Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ConcurrencyLimit::ResumeWorker JID-bbea76fdc97a6b8f611168e3: done: 0.528721 sec","job_status":"done","queue_duration_s":0.326123,"scheduling_latency_s":0.326147,"redis_calls":564,"redis_duration_s":0.176384,"redis_read_bytes":562,"redis_write_bytes":49063,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.002566,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004762,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":470,"redis_shared_state_calls":557,"redis_shared_state_duration_s":0.169056,"redis_shared_state_read_bytes":556,"redis_shared_state_write_bytes":48290,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.389336,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.528721,"completed_at":"2026-10-02T16:59:30.683Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003665,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2117gitlab # [ 415.813644] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.685Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d4efd91264a10b4cec8eef07","created_at":"2026-10-02T16:59:29.855Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.869Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d4efd91264a10b4cec8eef07: start","job_status":"start","queue_duration_s":0.815748,"scheduling_latency_s":0.815775}2118gitlab # [ 415.875482] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.811Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","jid":"d4efd91264a10b4cec8eef07","created_at":"2026-10-02T16:59:29.855Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:96987dbff74cf5606dd015c238540c0a2d4ff0cd1d1c93a64fda0dd15251e306","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.869Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::ScheduleUnlockPipelinesInQueueCronWorker JID-d4efd91264a10b4cec8eef07: done: 0.126436 sec","job_status":"done","queue_duration_s":0.815748,"scheduling_latency_s":0.815775,"redis_calls":70,"redis_duration_s":0.012624999999999999,"redis_read_bytes":118,"redis_write_bytes":42580,"redis_feature_flag_calls":10,"redis_feature_flag_duration_s":0.001617,"redis_feature_flag_read_bytes":10,"redis_feature_flag_write_bytes":1471,"redis_queues_calls":2,"redis_queues_duration_s":0.000602,"redis_queues_read_bytes":3,"redis_queues_write_bytes":37485,"redis_queues_metadata_calls":55,"redis_queues_metadata_duration_s":0.009888,"redis_queues_metadata_read_bytes":104,"redis_queues_metadata_write_bytes":3324,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000518,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":300,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.013,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.102114,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.126436,"completed_at":"2026-10-02T16:59:30.811Z","load_balancing_strategy":"primary","db_duration_s":0.013126,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2119gitlab # [ 415.943774] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:30.813Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6a9d1fb5a81953212316e79c","created_at":"2026-10-02T16:59:29.888Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcd4e46f54ec70f9e99e3d2d291c7634","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.898Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6a9d1fb5a81953212316e79c: start","job_status":"start","queue_duration_s":0.914702,"scheduling_latency_s":0.91472}2120gitlab # [ 416.119889] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.056Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CleanupWorker","jid":"6a9d1fb5a81953212316e79c","created_at":"2026-10-02T16:59:29.888Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcd4e46f54ec70f9e99e3d2d291c7634","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:2180384c8554b410820afa81ad3479301139c7627524f882c6bca44488742df9","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.898Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CleanupWorker JID-6a9d1fb5a81953212316e79c: done: 0.242897 sec","job_status":"done","queue_duration_s":0.914702,"scheduling_latency_s":0.91472,"redis_calls":13,"redis_duration_s":0.00251,"redis_read_bytes":11,"redis_write_bytes":1563,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.000661,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":576,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00097,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":476,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000879,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":511,"db_count":131,"db_write_count":0,"db_cached_count":84,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":131,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":84,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.037,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":129,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":84,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.035,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.195229,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00058386,"exclusive_lock_hold_duration_s":0.21845282,"extra.loose_foreign_keys_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"connection":"ci","turbo_mode":false},"duration_s":0.242897,"completed_at":"2026-10-02T16:59:31.056Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.052964,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2121gitlab # [ 416.189501] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.057Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3a0d56b035e196d37c577168","created_at":"2026-10-02T16:59:29.903Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea65cb66de3892187baf8f3533fc02b9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.922Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3a0d56b035e196d37c577168: start","job_status":"start","queue_duration_s":1.135239,"scheduling_latency_s":1.135258}2122gitlab # [ 416.207616] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.108Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker","jid":"3a0d56b035e196d37c577168","created_at":"2026-10-02T16:59:29.903Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea65cb66de3892187baf8f3533fc02b9","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:10c3834637b4f27873c5f81cc6d8cb3cf1692f0fbee5628a0a3fe91dc0dcbc57","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.922Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::MergeRequestDiffCommitCleanupWorker JID-3a0d56b035e196d37c577168: done: 0.051076 sec","job_status":"done","queue_duration_s":1.135239,"scheduling_latency_s":1.135258,"redis_calls":11,"redis_duration_s":0.002937,"redis_read_bytes":9,"redis_write_bytes":1518,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000461,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":349,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001286,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":554,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.00119,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":615,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.022,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":6,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.022,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021362,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00085224,"exclusive_lock_hold_duration_s":0.01768172,"extra.loose_foreign_keys_merge_request_diff_commit_cleanup_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0},"duration_s":0.051076,"completed_at":"2026-10-02T16:59:31.108Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.022966,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2123gitlab # [ 416.255275] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.110Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7e24bf931cd0c04423e464e4","created_at":"2026-10-02T16:59:29.934Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a873337cf7e0019fe25eb5b5c1492684","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.950Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7e24bf931cd0c04423e464e4: start","job_status":"start","queue_duration_s":1.159764,"scheduling_latency_s":1.159783}2124gitlab # [ 416.270080] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.136Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"UserStatusCleanup::BatchWorker","jid":"7e24bf931cd0c04423e464e4","created_at":"2026-10-02T16:59:29.934Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"a873337cf7e0019fe25eb5b5c1492684","meta.root_caller_id":"Cronjob","meta.feature_category":"user_profile","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63596f84f2aa01599e5d284b58c5b497d0d152817e7aa09bae73984558ee32bf","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.950Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"UserStatusCleanup::BatchWorker JID-7e24bf931cd0c04423e464e4: done: 0.025995 sec","job_status":"done","queue_duration_s":1.159764,"scheduling_latency_s":1.159783,"redis_calls":9,"redis_duration_s":0.00176,"redis_read_bytes":7,"redis_write_bytes":984,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000378,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":303,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":473,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000382,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":208,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015467,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025995,"completed_at":"2026-10-02T16:59:31.136Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004026,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2125gitlab # [ 416.314242] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.137Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a2fed7c5224d77e31cf877ca","created_at":"2026-10-02T16:59:29.964Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b1ea9f269eedf378cbf33f0281e678d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.975Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a2fed7c5224d77e31cf877ca: start","job_status":"start","queue_duration_s":1.162114,"scheduling_latency_s":1.162132}2126gitlab # [ 416.329275] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.148Z","class":"Database::BackgroundOperation::CiSchedulerWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerWorker execution for database that shares database configuration with another database","retry":0}2127gitlab # [ 416.332872] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.155Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerWorker","jid":"a2fed7c5224d77e31cf877ca","created_at":"2026-10-02T16:59:29.964Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"4b1ea9f269eedf378cbf33f0281e678d","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:4b61026b1cc556e95b4252138194cc7290e8501b85e4e3c9de25155a8ef9d1ac","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.975Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerWorker JID-a2fed7c5224d77e31cf877ca: done: 0.01835 sec","job_status":"done","queue_duration_s":1.162114,"scheduling_latency_s":1.162132,"redis_calls":9,"redis_duration_s":0.00158,"redis_read_bytes":7,"redis_write_bytes":1105,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000363,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":339,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000865,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":524,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000352,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":242,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011347,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.01835,"completed_at":"2026-10-02T16:59:31.155Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000443,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2128gitlab # [ 416.376596] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.157Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d115aafb362493bb857fabf1","created_at":"2026-10-02T16:59:29.981Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d526f2ca340b17bbefd4efadf295460","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.990Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d115aafb362493bb857fabf1: start","job_status":"start","queue_duration_s":1.166598,"scheduling_latency_s":1.166617}2129gitlab # [ 416.391643] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.168Z","class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","database":"sec","message":"Skipping Database::BackgroundOperation::SecSchedulerCellLocalWorker execution for unconfigured database sec","retry":0}2130gitlab # [ 416.395333] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.175Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::SecSchedulerCellLocalWorker","jid":"d115aafb362493bb857fabf1","created_at":"2026-10-02T16:59:29.981Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"6d526f2ca340b17bbefd4efadf295460","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:537c22ffe7dac982b6d86e5b8becfbed4376cc52983c5f1974bcd576563bd73a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:29.990Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::SecSchedulerCellLocalWorker JID-d115aafb362493bb857fabf1: done: 0.018 sec","job_status":"done","queue_duration_s":1.166598,"scheduling_latency_s":1.166617,"redis_calls":9,"redis_duration_s":0.001573,"redis_read_bytes":7,"redis_write_bytes":1185,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000348,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":359,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000889,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000336,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011092,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.018,"completed_at":"2026-10-02T16:59:31.175Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000355,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2131gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2132gitlab # [ 416.439530] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.176Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1b675a9446e613d046c3345a","created_at":"2026-10-02T16:59:30.003Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.018Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1b675a9446e613d046c3345a: start","job_status":"start","queue_duration_s":1.15792,"scheduling_latency_s":1.157937}2133gitlab # [ 416.452996] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.211Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"ScheduleMergeRequestCleanupRefsWorker","jid":"1b675a9446e613d046c3345a","created_at":"2026-10-02T16:59:30.003Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:33e8a9dcd4c9780ad0ea123ad7ccbabde1aa1e90ffcbb928434ba4b5800a5811","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.018Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"ScheduleMergeRequestCleanupRefsWorker JID-1b675a9446e613d046c3345a: done: 0.035303 sec","job_status":"done","queue_duration_s":1.15792,"scheduling_latency_s":1.157937,"redis_calls":18,"redis_duration_s":0.003601,"redis_read_bytes":20,"redis_write_bytes":4698,"redis_feature_flag_calls":4,"redis_feature_flag_duration_s":0.00074,"redis_feature_flag_read_bytes":4,"redis_feature_flag_write_bytes":672,"redis_queues_calls":2,"redis_queues_duration_s":0.000589,"redis_queues_read_bytes":3,"redis_queues_write_bytes":3015,"redis_queues_metadata_calls":9,"redis_queues_metadata_duration_s":0.001607,"redis_queues_metadata_read_bytes":12,"redis_queues_metadata_write_bytes":727,"redis_shared_state_calls":3,"redis_shared_state_duration_s":0.000665,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":284,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.02555,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035303,"completed_at":"2026-10-02T16:59:31.211Z","load_balancing_strategy":"primary","db_duration_s":0.002035,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2134gitlab # [ 416.497091] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.212Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a4ce36c8d76cae1d8b582152","created_at":"2026-10-02T16:59:30.030Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcce5a8937ef0f6a33a5eef08774c5d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:59:30.048Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a4ce36c8d76cae1d8b582152: start","job_status":"start","queue_duration_s":1.163993,"scheduling_latency_s":1.16401}2135gitlab # [ 416.511020] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.332Z","class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","database":"ci","message":"Skipping Database::BatchedBackgroundMigration::CiDatabaseWorker execution for database that shares database configuration with another database","retry":0}2136gitlab # [ 416.514892] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.341Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::CiDatabaseWorker","jid":"a4ce36c8d76cae1d8b582152","created_at":"2026-10-02T16:59:30.030Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"fcce5a8937ef0f6a33a5eef08774c5d7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:6ba8adee4a8c1e77d2f087a2765c43226ceffa1fd65abc34b95725a7c9abd857","enqueued_at":"2026-10-02T16:59:30.048Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::CiDatabaseWorker JID-a4ce36c8d76cae1d8b582152: done: 0.128097 sec","job_status":"done","queue_duration_s":1.163993,"scheduling_latency_s":1.16401,"redis_calls":9,"redis_duration_s":0.0017989999999999998,"redis_read_bytes":7,"redis_write_bytes":1152,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000464,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":351,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000986,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":545,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000349,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":256,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.12063,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.128097,"completed_at":"2026-10-02T16:59:31.340Z","load_balancing_strategy":"primary","db_duration_s":0.000406,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2137gitlab # [ 416.558912] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.342Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9347dd1ad3548525591aaa6f","created_at":"2026-10-02T16:59:30.059Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"669f6f481d995a6e354c53bebcd8a999","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.069Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9347dd1ad3548525591aaa6f: start","job_status":"start","queue_duration_s":1.272711,"scheduling_latency_s":1.272731}2138gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.16 seconds)2139gitlab # [ 416.573570] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.378Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker","jid":"9347dd1ad3548525591aaa6f","created_at":"2026-10-02T16:59:30.059Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"669f6f481d995a6e354c53bebcd8a999","meta.root_caller_id":"Cronjob","meta.feature_category":"groups_and_projects","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:b794e71d9c486ce6fd40978731717a0676e95bc1365a385a9a037f2f3c02ae2a","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.069Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Namespaces::ProcessOutdatedNamespaceDescendantsCronWorker JID-9347dd1ad3548525591aaa6f: done: 0.035879 sec","job_status":"done","queue_duration_s":1.272711,"scheduling_latency_s":1.272731,"redis_calls":9,"redis_duration_s":0.002304,"redis_read_bytes":7,"redis_write_bytes":1183,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000346,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.00155,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":560,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000408,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":266,"db_count":2,"db_write_count":1,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.009,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.009,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016597,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.namespaces_process_outdated_namespace_descendants_cron_worker.result":{},"duration_s":0.035879,"completed_at":"2026-10-02T16:59:31.378Z","load_balancing_strategy":"primary","db_duration_s":0.01145,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2140gitlab # [ 416.617619] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.379Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4828ad85e10d251ff907b6c0","created_at":"2026-10-02T16:59:30.094Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df3f66a4820a217af73637565a3b22a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:59:30.107Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4828ad85e10d251ff907b6c0: start","job_status":"start","queue_duration_s":1.271524,"scheduling_latency_s":1.271542}2141gitlab # [ 416.631554] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.391Z","class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","database":"sec","message":"Skipping Database::BatchedBackgroundMigration::SecDatabaseWorker execution for unconfigured database sec","retry":0}2142gitlab # [ 416.635567] gitaly[916]: time="2026-10-02T16:59:31.498Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX1PV0QYJNFG1CW8A0547 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:01.097" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:31.498" grpc.time_ms=0.669 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2143gitlab # [ 416.649181] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:31 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1122144gitlab # [ 416.651884] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.400Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigration::SecDatabaseWorker","jid":"4828ad85e10d251ff907b6c0","created_at":"2026-10-02T16:59:30.094Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"df3f66a4820a217af73637565a3b22a7","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:73dac367f8734ec768b767029c45c4e77750d97eb75e9dc81918dee7e9146dd0","enqueued_at":"2026-10-02T16:59:30.107Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigration::SecDatabaseWorker JID-4828ad85e10d251ff907b6c0: done: 0.021515 sec","job_status":"done","queue_duration_s":1.271524,"scheduling_latency_s":1.271542,"redis_calls":9,"redis_duration_s":0.002261,"redis_read_bytes":7,"redis_write_bytes":1159,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000413,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":353,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001422,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":548,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000426,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":258,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.001,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.021515,"completed_at":"2026-10-02T16:59:31.400Z","load_balancing_strategy":"primary","db_duration_s":0.000533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2145gitlab # [ 416.695207] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.402Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"aa1438bc9a09f2acec512c81","created_at":"2026-10-02T16:59:30.121Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d318c6c1cd40f6edf1567f38b8fc589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.143Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-aa1438bc9a09f2acec512c81: start","job_status":"start","queue_duration_s":1.258641,"scheduling_latency_s":1.258676}2146gitlab # [ 416.710466] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.458Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker","jid":"aa1438bc9a09f2acec512c81","created_at":"2026-10-02T16:59:30.121Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"9d318c6c1cd40f6edf1567f38b8fc589","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:e003caf073151f1134b113654ee6c14127eb98e5e1dd1956c06ead380cd79bba","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.143Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"LooseForeignKeys::CiPipelinesBuildsCleanupCronWorker JID-aa1438bc9a09f2acec512c81: done: 0.055983 sec","job_status":"done","queue_duration_s":1.258641,"scheduling_latency_s":1.258676,"redis_calls":11,"redis_duration_s":0.0018839999999999998,"redis_read_bytes":9,"redis_write_bytes":1509,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000309,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":347,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000859,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":551,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000716,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":611,"db_count":7,"db_write_count":0,"db_cached_count":2,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":2,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.02,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":2,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.001,"db_ci_duration_s":0.02,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.026196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0.00053208,"exclusive_lock_hold_duration_s":0.03285368,"extra.loose_foreign_keys_ci_pipelines_builds_cleanup_cron_worker.stats":{"over_limit":false,"delete_count_by_table":{},"update_count_by_table":{},"delete_count":0,"update_count":0,"turbo_mode":false},"duration_s":0.055983,"completed_at":"2026-10-02T16:59:31.458Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.021995,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2147gitlab # [ 416.759143] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.459Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0efc32f7deb9b4b0e71447b7","created_at":"2026-10-02T16:59:30.151Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c5591db52d0ade476c14c5bd0511e33","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.162Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0efc32f7deb9b4b0e71447b7: start","job_status":"start","queue_duration_s":1.297587,"scheduling_latency_s":1.297607}2148gitlab # [ 416.774261] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.492Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Ci::Catalog::Resources::ProcessSyncEventsWorker","jid":"0efc32f7deb9b4b0e71447b7","created_at":"2026-10-02T16:59:30.151Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"1c5591db52d0ade476c14c5bd0511e33","meta.root_caller_id":"Cronjob","meta.feature_category":"pipeline_composition","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:bdea4bf4c16667cd22a226e829c85f9746ee8d058f39e317ef84659d4044477c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.162Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::Catalog::Resources::ProcessSyncEventsWorker JID-0efc32f7deb9b4b0e71447b7: done: 0.033003 sec","job_status":"done","queue_duration_s":1.297587,"scheduling_latency_s":1.297607,"redis_calls":12,"redis_duration_s":0.002539,"redis_read_bytes":10,"redis_write_bytes":1576,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.00041,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":337,"redis_queues_metadata_calls":6,"redis_queues_metadata_duration_s":0.00123,"redis_queues_metadata_read_bytes":5,"redis_queues_metadata_write_bytes":636,"redis_shared_state_calls":4,"redis_shared_state_duration_s":0.000899,"redis_shared_state_read_bytes":3,"redis_shared_state_write_bytes":603,"db_count":4,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021944,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.00986224,"extra.ci_catalog_resources_process_sync_events_worker.estimated_total_events":0,"extra.ci_catalog_resources_process_sync_events_worker.consumable_events":0,"duration_s":0.033003,"completed_at":"2026-10-02T16:59:31.492Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.003103,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2149gitlab # [ 416.820481] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.494Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7b8eae3face660101fa651a7","created_at":"2026-10-02T16:59:30.183Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abc2faf450879e34e316b5ffac1edcf6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.195Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7b8eae3face660101fa651a7: start","job_status":"start","queue_duration_s":1.298287,"scheduling_latency_s":1.298304}2150gitlab # [ 416.835245] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.522Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Cells::LostTransactionRecoveryWorker","jid":"7b8eae3face660101fa651a7","created_at":"2026-10-02T16:59:30.183Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"abc2faf450879e34e316b5ffac1edcf6","meta.root_caller_id":"Cronjob","meta.feature_category":"cell","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:0fd4def8dd5675dcd84c556c4d40bc905cecfdb470c7c1510e8a70079f5c8a13","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.195Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Cells::LostTransactionRecoveryWorker JID-7b8eae3face660101fa651a7: done: 0.028239 sec","job_status":"done","queue_duration_s":1.298287,"scheduling_latency_s":1.298304,"redis_calls":9,"redis_duration_s":0.001836,"redis_read_bytes":7,"redis_write_bytes":1026,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000359,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":315,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001054,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":491,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000423,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":220,"db_count":3,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01657,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.028239,"completed_at":"2026-10-02T16:59:31.522Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.004478,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2151gitlab # [ 416.878798] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.523Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8d83d377f5f75ee97ef21168","created_at":"2026-10-02T16:59:30.211Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3190768c946ac64d6b5347bfa9e6fd22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:59:30.224Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8d83d377f5f75ee97ef21168: start","job_status":"start","queue_duration_s":1.299365,"scheduling_latency_s":1.299383}2152gitlab # [ 416.892754] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.559Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BatchedBackgroundMigrationWorker","jid":"8d83d377f5f75ee97ef21168","created_at":"2026-10-02T16:59:30.211Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"3190768c946ac64d6b5347bfa9e6fd22","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:592d9619e1997b640b70ce6a22f6713bc7793bb7a4e342b7380d90b691fcd6ae","enqueued_at":"2026-10-02T16:59:30.224Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BatchedBackgroundMigrationWorker JID-8d83d377f5f75ee97ef21168: done: 0.035598 sec","job_status":"done","queue_duration_s":1.299365,"scheduling_latency_s":1.299383,"redis_calls":9,"redis_duration_s":0.002596,"redis_read_bytes":7,"redis_write_bytes":1068,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.001071,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":327,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001064,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000461,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":2,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":2,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.004,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.004,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.022143,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.035598,"completed_at":"2026-10-02T16:59:31.559Z","load_balancing_strategy":"primary","db_duration_s":0.006533,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2153gitlab # [ 416.934929] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.561Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d30912a4ffd8dc27c1db6d11","created_at":"2026-10-02T16:59:30.248Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea141a85ce179afc88dffec6f8bc2e45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.285Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d30912a4ffd8dc27c1db6d11: start","job_status":"start","queue_duration_s":1.276089,"scheduling_latency_s":1.276108}2154gitlab # [ 416.950086] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.573Z","class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","database":"ci","message":"Skipping Database::BackgroundOperation::CiSchedulerCellLocalWorker execution for database that shares database configuration with another database","retry":0}2155gitlab # [ 416.954316] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.581Z","retry":0,"queue":"default","version":0,"store":null,"queue_namespace":"cronjob","args":[],"class":"Database::BackgroundOperation::CiSchedulerCellLocalWorker","jid":"d30912a4ffd8dc27c1db6d11","created_at":"2026-10-02T16:59:30.248Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-b35adb9d363d4f2e","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-environment=,sentry-release=Unknown"},"meta.caller_id":"Cronjob","correlation_id":"ea141a85ce179afc88dffec6f8bc2e45","meta.root_caller_id":"Cronjob","meta.feature_category":"database","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8b555168ccdb6ae74d10de2e5fe446d2cd23b814df22448d98e7a3685b03eba6","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.285Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Database::BackgroundOperation::CiSchedulerCellLocalWorker JID-d30912a4ffd8dc27c1db6d11: done: 0.019784 sec","job_status":"done","queue_duration_s":1.276089,"scheduling_latency_s":1.276108,"redis_calls":9,"redis_duration_s":0.0016129999999999999,"redis_read_bytes":7,"redis_write_bytes":1178,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000365,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":357,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":557,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000339,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":264,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011994,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.019784,"completed_at":"2026-10-02T16:59:31.581Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.000455,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2156gitlab # [ 416.999449] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.582Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"847a6092df708fb985edbe1b","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-847a6092df708fb985edbe1b: start","job_status":"start","queue_duration_s":0.77926,"scheduling_latency_s":0.779276}2157gitlab # [ 417.012800] nix-daemon[3641]: accepted connection from pid 4768, user nixbot2158gitlab # [ 417.013771] nix-daemon[3641]: accepted connection from pid 4731, user nixbot2159gitlab # [ 417.014794] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.604Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"847a6092df708fb985edbe1b","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-847a6092df708fb985edbe1b: done: 0.021917 sec","job_status":"done","queue_duration_s":0.77926,"scheduling_latency_s":0.779276,"redis_calls":15,"redis_duration_s":0.002546,"redis_read_bytes":12,"redis_write_bytes":1464,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000348,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":307,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000687,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001511,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":1,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":1,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013334,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021917,"completed_at":"2026-10-02T16:59:31.604Z","load_balancing_strategy":"primary","db_duration_s":0.000375,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2160gitlab # [ 417.057339] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.605Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab00532b795f27fad9a525a7","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab00532b795f27fad9a525a7: start","job_status":"start","queue_duration_s":0.802389,"scheduling_latency_s":0.802406}2161gitlab # [ 417.070120] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.621Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ab00532b795f27fad9a525a7","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ab00532b795f27fad9a525a7: done: 0.015968 sec","job_status":"done","queue_duration_s":0.802389,"scheduling_latency_s":0.802406,"redis_calls":13,"redis_duration_s":0.002665,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001373,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001292,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010715,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015968,"completed_at":"2026-10-02T16:59:31.621Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2162gitlab # [ 417.110680] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.622Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"064ed64cebdd1266ae7b839b","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-064ed64cebdd1266ae7b839b: start","job_status":"start","queue_duration_s":0.819548,"scheduling_latency_s":0.819565}2163gitlab # [ 417.123640] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.639Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"064ed64cebdd1266ae7b839b","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-064ed64cebdd1266ae7b839b: done: 0.016568 sec","job_status":"done","queue_duration_s":0.819548,"scheduling_latency_s":0.819565,"redis_calls":13,"redis_duration_s":0.001977,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001323,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010751,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016568,"completed_at":"2026-10-02T16:59:31.639Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2164gitlab # [ 417.164199] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.640Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3fa9a072fb9f1c8df00d654","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3fa9a072fb9f1c8df00d654: start","job_status":"start","queue_duration_s":0.837333,"scheduling_latency_s":0.837349}2165gitlab # [ 417.176983] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.656Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f3fa9a072fb9f1c8df00d654","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f3fa9a072fb9f1c8df00d654: done: 0.015958 sec","job_status":"done","queue_duration_s":0.837333,"scheduling_latency_s":0.837349,"redis_calls":13,"redis_duration_s":0.001914,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000641,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001273,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011059,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015958,"completed_at":"2026-10-02T16:59:31.656Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2166gitlab # [ 417.218190] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.658Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa7b1efab3de1b541a7b004","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa7b1efab3de1b541a7b004: start","job_status":"start","queue_duration_s":0.855231,"scheduling_latency_s":0.855254}2167gitlab # [ 417.231103] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.676Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6fa7b1efab3de1b541a7b004","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6fa7b1efab3de1b541a7b004: done: 0.018739 sec","job_status":"done","queue_duration_s":0.855231,"scheduling_latency_s":0.855254,"redis_calls":13,"redis_duration_s":0.0018970000000000002,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000671,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001226,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018739,"completed_at":"2026-10-02T16:59:31.676Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2168gitlab # [ 417.271825] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.677Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cbbcb81476aac652cd6d996","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cbbcb81476aac652cd6d996: start","job_status":"start","queue_duration_s":0.874434,"scheduling_latency_s":0.87445}2169gitlab # [ 417.284814] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.693Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"3cbbcb81476aac652cd6d996","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-3cbbcb81476aac652cd6d996: done: 0.015901 sec","job_status":"done","queue_duration_s":0.874434,"scheduling_latency_s":0.87445,"redis_calls":13,"redis_duration_s":0.001988,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000684,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001304,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010953,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015901,"completed_at":"2026-10-02T16:59:31.693Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2170gitlab # [ 417.325010] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.694Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1305ea911ff444105a24ce8d","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1305ea911ff444105a24ce8d: start","job_status":"start","queue_duration_s":0.891522,"scheduling_latency_s":0.891538}2171gitlab # [ 417.337805] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.712Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1305ea911ff444105a24ce8d","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1305ea911ff444105a24ce8d: done: 0.017294 sec","job_status":"done","queue_duration_s":0.891522,"scheduling_latency_s":0.891538,"redis_calls":13,"redis_duration_s":0.002144,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000851,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001293,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011106,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017294,"completed_at":"2026-10-02T16:59:31.712Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2172gitlab # [ 417.378212] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.713Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6ba786c963401262f792df6","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6ba786c963401262f792df6: start","job_status":"start","queue_duration_s":0.910034,"scheduling_latency_s":0.910051}2173gitlab # [ 417.391230] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.729Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c6ba786c963401262f792df6","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c6ba786c963401262f792df6: done: 0.015989 sec","job_status":"done","queue_duration_s":0.910034,"scheduling_latency_s":0.910051,"redis_calls":13,"redis_duration_s":0.002098,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000704,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001394,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011051,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015989,"completed_at":"2026-10-02T16:59:31.729Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2174gitlab # [ 417.432956] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.731Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fd25eeea3aa14fac5f90c3e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fd25eeea3aa14fac5f90c3e: start","job_status":"start","queue_duration_s":0.928241,"scheduling_latency_s":0.92826}2175gitlab # [ 417.445668] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.747Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"9fd25eeea3aa14fac5f90c3e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-9fd25eeea3aa14fac5f90c3e: done: 0.015913 sec","job_status":"done","queue_duration_s":0.928241,"scheduling_latency_s":0.92826,"redis_calls":13,"redis_duration_s":0.001911,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000674,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001237,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010997,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015913,"completed_at":"2026-10-02T16:59:31.747Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2176gitlab # [ 417.486029] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.748Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf4a54412034c69301a6ff3","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf4a54412034c69301a6ff3: start","job_status":"start","queue_duration_s":0.945304,"scheduling_latency_s":0.94532}2177gitlab # [ 417.498873] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.766Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8cf4a54412034c69301a6ff3","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8cf4a54412034c69301a6ff3: done: 0.017623 sec","job_status":"done","queue_duration_s":0.945304,"scheduling_latency_s":0.94532,"redis_calls":13,"redis_duration_s":0.0019979999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000749,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001249,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01276,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017623,"completed_at":"2026-10-02T16:59:31.766Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2178gitlab # [ 417.539557] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.767Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71e978fe33d3cf04f6ceb7ea","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71e978fe33d3cf04f6ceb7ea: start","job_status":"start","queue_duration_s":0.96418,"scheduling_latency_s":0.964197}2179gitlab # [ 417.552654] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.783Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"71e978fe33d3cf04f6ceb7ea","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-71e978fe33d3cf04f6ceb7ea: done: 0.015494 sec","job_status":"done","queue_duration_s":0.96418,"scheduling_latency_s":0.964197,"redis_calls":13,"redis_duration_s":0.001765,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000672,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001093,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010806,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015494,"completed_at":"2026-10-02T16:59:31.782Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2180gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2181gitlab # [ 417.593182] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.785Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440517826ce01d8b4c9e9ac0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440517826ce01d8b4c9e9ac0: start","job_status":"start","queue_duration_s":0.981819,"scheduling_latency_s":0.981836}2182gitlab # [ 417.605852] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.802Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"440517826ce01d8b4c9e9ac0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-440517826ce01d8b4c9e9ac0: done: 0.017736 sec","job_status":"done","queue_duration_s":0.981819,"scheduling_latency_s":0.981836,"redis_calls":13,"redis_duration_s":0.002589,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001912,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012896,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017736,"completed_at":"2026-10-02T16:59:31.802Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2183gitlab # [ 417.645875] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.804Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8da7f68f6eef45291df4d5da","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8da7f68f6eef45291df4d5da: start","job_status":"start","queue_duration_s":1.001046,"scheduling_latency_s":1.001064}2184gitlab # [ 417.658831] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.821Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8da7f68f6eef45291df4d5da","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8da7f68f6eef45291df4d5da: done: 0.016963 sec","job_status":"done","queue_duration_s":1.001046,"scheduling_latency_s":1.001064,"redis_calls":13,"redis_duration_s":0.001687,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001053,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011767,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016963,"completed_at":"2026-10-02T16:59:31.821Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2185gitlab # [ 417.699495] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.823Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"952bee658a21d367a74ae766","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-952bee658a21d367a74ae766: start","job_status":"start","queue_duration_s":1.019612,"scheduling_latency_s":1.019631}2186gitlab # [ 417.712595] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.838Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"952bee658a21d367a74ae766","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-952bee658a21d367a74ae766: done: 0.015398 sec","job_status":"done","queue_duration_s":1.019612,"scheduling_latency_s":1.019631,"redis_calls":13,"redis_duration_s":0.001515,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000554,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000961,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010857,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015398,"completed_at":"2026-10-02T16:59:31.838Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2187gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.16 seconds)2188gitlab # [ 417.753223] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.839Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844b9782858cdd4e446ea689","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844b9782858cdd4e446ea689: start","job_status":"start","queue_duration_s":1.036102,"scheduling_latency_s":1.036119}2189gitlab # [ 417.766366] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.854Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"844b9782858cdd4e446ea689","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-844b9782858cdd4e446ea689: done: 0.015455 sec","job_status":"done","queue_duration_s":1.036102,"scheduling_latency_s":1.036119,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000573,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000936,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010513,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015455,"completed_at":"2026-10-02T16:59:31.854Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2190gitlab # [ 417.806821] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.856Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29f6aabb2aff2316b19913e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29f6aabb2aff2316b19913e: start","job_status":"start","queue_duration_s":1.052649,"scheduling_latency_s":1.052667}2191gitlab # [ 417.820302] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:32 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1032192gitlab # [ 417.823382] gitaly[916]: time="2026-10-02T16:59:32.660Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX2VDY73FAX8DWBTAVR23 grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:02.259" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:32.659" grpc.time_ms=0.819 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2193gitlab # [ 417.836655] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.871Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f29f6aabb2aff2316b19913e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f29f6aabb2aff2316b19913e: done: 0.01526 sec","job_status":"done","queue_duration_s":1.052649,"scheduling_latency_s":1.052667,"redis_calls":13,"redis_duration_s":0.001591,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000601,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00099,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010722,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01526,"completed_at":"2026-10-02T16:59:31.871Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2194gitlab # [ 417.877756] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.872Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2329cd2a016b887bf86b1294","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2329cd2a016b887bf86b1294: start","job_status":"start","queue_duration_s":1.069161,"scheduling_latency_s":1.069179}2195gitlab # [ 417.890559] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.887Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2329cd2a016b887bf86b1294","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2329cd2a016b887bf86b1294: done: 0.015358 sec","job_status":"done","queue_duration_s":1.069161,"scheduling_latency_s":1.069179,"redis_calls":13,"redis_duration_s":0.001509,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000536,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.000973,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010537,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015358,"completed_at":"2026-10-02T16:59:31.887Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2196gitlab # [ 417.931142] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.889Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d201850e319f12fd7ddd9dc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d201850e319f12fd7ddd9dc: start","job_status":"start","queue_duration_s":1.085658,"scheduling_latency_s":1.085676}2197gitlab # [ 417.944149] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.904Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"1d201850e319f12fd7ddd9dc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-1d201850e319f12fd7ddd9dc: done: 0.015256 sec","job_status":"done","queue_duration_s":1.085658,"scheduling_latency_s":1.085676,"redis_calls":13,"redis_duration_s":0.0014910000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000531,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00096,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01049,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015256,"completed_at":"2026-10-02T16:59:31.904Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2198gitlab # [ 417.984538] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.905Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"686d70aa671c1708d47e0d07","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-686d70aa671c1708d47e0d07: start","job_status":"start","queue_duration_s":1.102033,"scheduling_latency_s":1.10205}2199gitlab # [ 417.997163] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.921Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"686d70aa671c1708d47e0d07","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-686d70aa671c1708d47e0d07: done: 0.015605 sec","job_status":"done","queue_duration_s":1.102033,"scheduling_latency_s":1.10205,"redis_calls":13,"redis_duration_s":0.001777,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000545,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010874,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015605,"completed_at":"2026-10-02T16:59:31.920Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2200gitlab # [ 418.037480] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.922Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92d1e1119a48fbe9fa7a9cfb","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92d1e1119a48fbe9fa7a9cfb: start","job_status":"start","queue_duration_s":1.118774,"scheduling_latency_s":1.11879}2201gitlab # [ 418.050314] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.937Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"92d1e1119a48fbe9fa7a9cfb","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-92d1e1119a48fbe9fa7a9cfb: done: 0.015425 sec","job_status":"done","queue_duration_s":1.118774,"scheduling_latency_s":1.11879,"redis_calls":13,"redis_duration_s":0.0019320000000000001,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000909,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001023,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010455,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015425,"completed_at":"2026-10-02T16:59:31.937Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2202gitlab # [ 418.091645] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.938Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"637b6b6780b7316272a53d27","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-637b6b6780b7316272a53d27: start","job_status":"start","queue_duration_s":1.135394,"scheduling_latency_s":1.13541}2203gitlab # [ 418.104671] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.953Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"637b6b6780b7316272a53d27","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-637b6b6780b7316272a53d27: done: 0.015232 sec","job_status":"done","queue_duration_s":1.135394,"scheduling_latency_s":1.13541,"redis_calls":13,"redis_duration_s":0.0021130000000000003,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000737,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001376,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010196,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015232,"completed_at":"2026-10-02T16:59:31.953Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2204gitlab # [ 418.145348] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.955Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8960474b32fbf62e1a0b556e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8960474b32fbf62e1a0b556e: start","job_status":"start","queue_duration_s":1.151838,"scheduling_latency_s":1.151854}2205gitlab # [ 418.158107] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.970Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8960474b32fbf62e1a0b556e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8960474b32fbf62e1a0b556e: done: 0.015251 sec","job_status":"done","queue_duration_s":1.151838,"scheduling_latency_s":1.151854,"redis_calls":13,"redis_duration_s":0.001912,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000696,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001216,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01022,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015251,"completed_at":"2026-10-02T16:59:31.970Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2206gitlab # [ 418.198451] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.971Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b8dd4e3a5880c7344bcdfda","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b8dd4e3a5880c7344bcdfda: start","job_status":"start","queue_duration_s":1.168305,"scheduling_latency_s":1.168321}2207gitlab # [ 418.211285] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.986Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"7b8dd4e3a5880c7344bcdfda","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-7b8dd4e3a5880c7344bcdfda: done: 0.015063 sec","job_status":"done","queue_duration_s":1.168305,"scheduling_latency_s":1.168321,"redis_calls":13,"redis_duration_s":0.002038,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.0007,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001338,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015063,"completed_at":"2026-10-02T16:59:31.986Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2208gitlab # [ 418.251901] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:31.988Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f193ba496498798109240a50","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f193ba496498798109240a50: start","job_status":"start","queue_duration_s":1.184943,"scheduling_latency_s":1.184961}2209gitlab # [ 418.264777] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.003Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"f193ba496498798109240a50","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-f193ba496498798109240a50: done: 0.015087 sec","job_status":"done","queue_duration_s":1.184943,"scheduling_latency_s":1.184961,"redis_calls":13,"redis_duration_s":0.001989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000669,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00132,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010125,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015087,"completed_at":"2026-10-02T16:59:32.003Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2210gitlab # [ 418.305896] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.004Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58fe9aaf2421906551beb64a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58fe9aaf2421906551beb64a: start","job_status":"start","queue_duration_s":1.201178,"scheduling_latency_s":1.201194}2211gitlab # [ 418.318808] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.019Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"58fe9aaf2421906551beb64a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-58fe9aaf2421906551beb64a: done: 0.01491 sec","job_status":"done","queue_duration_s":1.201178,"scheduling_latency_s":1.201194,"redis_calls":13,"redis_duration_s":0.002053,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000934,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001119,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009965,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.01491,"completed_at":"2026-10-02T16:59:32.019Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2212gitlab # [ 418.359592] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.020Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8505ae43b683e2139c822bb7","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8505ae43b683e2139c822bb7: start","job_status":"start","queue_duration_s":1.217492,"scheduling_latency_s":1.217508}2213gitlab # [ 418.372479] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.036Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8505ae43b683e2139c822bb7","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8505ae43b683e2139c822bb7: done: 0.015345 sec","job_status":"done","queue_duration_s":1.217492,"scheduling_latency_s":1.217508,"redis_calls":13,"redis_duration_s":0.001969,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001256,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010375,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015345,"completed_at":"2026-10-02T16:59:32.036Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2214gitlab # [ 418.413028] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.037Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e42f6d8310b41cbd9b9e2fc9","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e42f6d8310b41cbd9b9e2fc9: start","job_status":"start","queue_duration_s":1.234074,"scheduling_latency_s":1.23409}2215gitlab # [ 418.425981] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.052Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"e42f6d8310b41cbd9b9e2fc9","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-e42f6d8310b41cbd9b9e2fc9: done: 0.015179 sec","job_status":"done","queue_duration_s":1.234074,"scheduling_latency_s":1.23409,"redis_calls":13,"redis_duration_s":0.001886,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000654,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010338,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015179,"completed_at":"2026-10-02T16:59:32.052Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2216gitlab # [ 418.466393] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.053Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69a23d3d4491c871e42321b0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69a23d3d4491c871e42321b0: start","job_status":"start","queue_duration_s":1.250402,"scheduling_latency_s":1.250419}2217gitlab # [ 418.479312] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.068Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"69a23d3d4491c871e42321b0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-69a23d3d4491c871e42321b0: done: 0.014872 sec","job_status":"done","queue_duration_s":1.250402,"scheduling_latency_s":1.250419,"redis_calls":13,"redis_duration_s":0.001948,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000634,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001314,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010038,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014872,"completed_at":"2026-10-02T16:59:32.068Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2218gitlab # [ 418.520898] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.069Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d9f3ff4f21401b41acb6cc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d9f3ff4f21401b41acb6cc: start","job_status":"start","queue_duration_s":1.26644,"scheduling_latency_s":1.266456}2219gitlab # [ 418.533743] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.084Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a1d9f3ff4f21401b41acb6cc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a1d9f3ff4f21401b41acb6cc: done: 0.015131 sec","job_status":"done","queue_duration_s":1.26644,"scheduling_latency_s":1.266456,"redis_calls":13,"redis_duration_s":0.0019909999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000681,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009954,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015131,"completed_at":"2026-10-02T16:59:32.084Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2220gitlab # [ 418.574464] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.086Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b21d7217da34c751e831188c","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b21d7217da34c751e831188c: start","job_status":"start","queue_duration_s":1.282746,"scheduling_latency_s":1.282762}2221gitlab # [ 418.587250] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.101Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b21d7217da34c751e831188c","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b21d7217da34c751e831188c: done: 0.014927 sec","job_status":"done","queue_duration_s":1.282746,"scheduling_latency_s":1.282762,"redis_calls":13,"redis_duration_s":0.001936,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00067,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001266,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010095,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014927,"completed_at":"2026-10-02T16:59:32.100Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2222gitlab # [ 418.628040] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.102Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee7a9de1a567a240c0223a7a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee7a9de1a567a240c0223a7a: start","job_status":"start","queue_duration_s":1.298857,"scheduling_latency_s":1.298873}2223gitlab # [ 418.640609] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.117Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ee7a9de1a567a240c0223a7a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ee7a9de1a567a240c0223a7a: done: 0.015599 sec","job_status":"done","queue_duration_s":1.298857,"scheduling_latency_s":1.298873,"redis_calls":13,"redis_duration_s":0.002236,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000695,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001541,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010139,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015599,"completed_at":"2026-10-02T16:59:32.117Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2224gitlab # [ 418.681033] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.119Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acef791dd9483e40fd7280c","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acef791dd9483e40fd7280c: start","job_status":"start","queue_duration_s":1.315661,"scheduling_latency_s":1.315678}2225gitlab # [ 418.694678] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.133Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8acef791dd9483e40fd7280c","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8acef791dd9483e40fd7280c: done: 0.014588 sec","job_status":"done","queue_duration_s":1.315661,"scheduling_latency_s":1.315678,"redis_calls":13,"redis_duration_s":0.001876,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000644,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001232,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.009826,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.014588,"completed_at":"2026-10-02T16:59:32.133Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2226gitlab # [ 418.735363] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.135Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9e0211981ea1e44a13bab9","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9e0211981ea1e44a13bab9: start","job_status":"start","queue_duration_s":1.331743,"scheduling_latency_s":1.33176}2227gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2228gitlab # [ 418.748294] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.153Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ea9e0211981ea1e44a13bab9","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ea9e0211981ea1e44a13bab9: done: 0.018094 sec","job_status":"done","queue_duration_s":1.331743,"scheduling_latency_s":1.33176,"redis_calls":13,"redis_duration_s":0.0023179999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000734,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001584,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012666,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018094,"completed_at":"2026-10-02T16:59:32.153Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2229gitlab # [ 418.787694] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.154Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"453e6431701c94a00649ccc2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-453e6431701c94a00649ccc2: start","job_status":"start","queue_duration_s":1.351371,"scheduling_latency_s":1.351394}2230gitlab # [ 418.800865] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.176Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"453e6431701c94a00649ccc2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-453e6431701c94a00649ccc2: done: 0.021825 sec","job_status":"done","queue_duration_s":1.351371,"scheduling_latency_s":1.351394,"redis_calls":13,"redis_duration_s":0.00217,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000762,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001408,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014912,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.021825,"completed_at":"2026-10-02T16:59:32.176Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2231gitlab # [ 418.841365] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.178Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01ae054c2083a5327a8029b6","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01ae054c2083a5327a8029b6: start","job_status":"start","queue_duration_s":1.3748,"scheduling_latency_s":1.374823}2232gitlab # [ 418.854219] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.199Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"01ae054c2083a5327a8029b6","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-01ae054c2083a5327a8029b6: done: 0.02134 sec","job_status":"done","queue_duration_s":1.3748,"scheduling_latency_s":1.374823,"redis_calls":13,"redis_duration_s":0.0027199999999999998,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001927,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014989,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02134,"completed_at":"2026-10-02T16:59:32.199Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2233gitlab # [ 418.894781] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.201Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"074c103f1ac2fc3f4114d1a8","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-074c103f1ac2fc3f4114d1a8: start","job_status":"start","queue_duration_s":1.39759,"scheduling_latency_s":1.397611}2234gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.17 seconds)2235gitlab # [ 418.908649] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.219Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"074c103f1ac2fc3f4114d1a8","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-074c103f1ac2fc3f4114d1a8: done: 0.018267 sec","job_status":"done","queue_duration_s":1.39759,"scheduling_latency_s":1.397611,"redis_calls":13,"redis_duration_s":0.002414,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000716,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001698,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.012301,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.018267,"completed_at":"2026-10-02T16:59:32.219Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2236gitlab # [ 418.949019] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.220Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a83841f666f9c114f89dc357","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a83841f666f9c114f89dc357: start","job_status":"start","queue_duration_s":1.41735,"scheduling_latency_s":1.417369}2237gitlab # [ 418.962135] gitaly[916]: time="2026-10-02T16:59:33.823Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX3ZPM420EY9PVJB4VPWC grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:03.421" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:33.822" grpc.time_ms=0.834 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2238gitlab # [ 418.975777] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:33 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1082239gitlab # [ 418.978685] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.238Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a83841f666f9c114f89dc357","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a83841f666f9c114f89dc357: done: 0.017814 sec","job_status":"done","queue_duration_s":1.41735,"scheduling_latency_s":1.417369,"redis_calls":13,"redis_duration_s":0.002159,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000744,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001415,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01215,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017814,"completed_at":"2026-10-02T16:59:32.238Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2240gitlab # [ 419.019356] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.240Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd562e16f5d501dd2a54d1e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd562e16f5d501dd2a54d1e: start","job_status":"start","queue_duration_s":1.436633,"scheduling_latency_s":1.436656}2241gitlab # [ 419.032369] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.257Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"fcd562e16f5d501dd2a54d1e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-fcd562e16f5d501dd2a54d1e: done: 0.017468 sec","job_status":"done","queue_duration_s":1.436633,"scheduling_latency_s":1.436656,"redis_calls":13,"redis_duration_s":0.002071,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000719,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001352,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01116,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017468,"completed_at":"2026-10-02T16:59:32.257Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2242gitlab # [ 419.072944] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.258Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bf27aa9e8cc85741148cb6f","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bf27aa9e8cc85741148cb6f: start","job_status":"start","queue_duration_s":1.455324,"scheduling_latency_s":1.455341}2243gitlab # [ 419.085827] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.274Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"6bf27aa9e8cc85741148cb6f","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-6bf27aa9e8cc85741148cb6f: done: 0.015362 sec","job_status":"done","queue_duration_s":1.455324,"scheduling_latency_s":1.455341,"redis_calls":13,"redis_duration_s":0.001987,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000677,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.00131,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010394,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.015362,"completed_at":"2026-10-02T16:59:32.274Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2244gitlab # [ 419.127206] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.275Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8785a26f7e0012c72afd51e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8785a26f7e0012c72afd51e: start","job_status":"start","queue_duration_s":1.472403,"scheduling_latency_s":1.47242}2245gitlab # [ 419.140188] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.296Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b8785a26f7e0012c72afd51e","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b8785a26f7e0012c72afd51e: done: 0.020777 sec","job_status":"done","queue_duration_s":1.472403,"scheduling_latency_s":1.47242,"redis_calls":13,"redis_duration_s":0.0033109999999999997,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000822,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.002489,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.015232,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020777,"completed_at":"2026-10-02T16:59:32.296Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2246gitlab # [ 419.180871] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.298Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed4760f07d142358da05cbc2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed4760f07d142358da05cbc2: start","job_status":"start","queue_duration_s":1.495432,"scheduling_latency_s":1.495457}2247gitlab # [ 419.193706] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.323Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"ed4760f07d142358da05cbc2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-ed4760f07d142358da05cbc2: done: 0.024449 sec","job_status":"done","queue_duration_s":1.495432,"scheduling_latency_s":1.495457,"redis_calls":13,"redis_duration_s":0.0029,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000923,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001977,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017182,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.024449,"completed_at":"2026-10-02T16:59:32.323Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2248gitlab # [ 419.234037] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.325Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aee9f1b16eed7d7cf25777a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aee9f1b16eed7d7cf25777a: start","job_status":"start","queue_duration_s":1.521595,"scheduling_latency_s":1.52162}2249gitlab # [ 419.246762] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.348Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"0aee9f1b16eed7d7cf25777a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-0aee9f1b16eed7d7cf25777a: done: 0.02308 sec","job_status":"done","queue_duration_s":1.521595,"scheduling_latency_s":1.52162,"redis_calls":13,"redis_duration_s":0.002338,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000793,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001545,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016117,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.02308,"completed_at":"2026-10-02T16:59:32.347Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2250gitlab # [ 419.287169] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.349Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d047d796c140087d7af7045","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d047d796c140087d7af7045: start","job_status":"start","queue_duration_s":1.546237,"scheduling_latency_s":1.546261}2251gitlab # [ 419.300111] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.374Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"2d047d796c140087d7af7045","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-2d047d796c140087d7af7045: done: 0.025369 sec","job_status":"done","queue_duration_s":1.546237,"scheduling_latency_s":1.546261,"redis_calls":13,"redis_duration_s":0.002807,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000971,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001836,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.017947,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025369,"completed_at":"2026-10-02T16:59:32.374Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2252gitlab # [ 419.341036] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.376Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4940652734fcf164caa260a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4940652734fcf164caa260a: start","job_status":"start","queue_duration_s":1.573179,"scheduling_latency_s":1.573202}2253gitlab # [ 419.353860] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.401Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"a4940652734fcf164caa260a","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-a4940652734fcf164caa260a: done: 0.025343 sec","job_status":"done","queue_duration_s":1.573179,"scheduling_latency_s":1.573202,"redis_calls":13,"redis_duration_s":0.002447,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000875,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001572,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.016607,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.025343,"completed_at":"2026-10-02T16:59:32.401Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2254gitlab # [ 419.394347] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.403Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8864fc73d7d0854b00553ad2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8864fc73d7d0854b00553ad2: start","job_status":"start","queue_duration_s":1.600098,"scheduling_latency_s":1.600123}2255gitlab # [ 419.407147] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.424Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"8864fc73d7d0854b00553ad2","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-8864fc73d7d0854b00553ad2: done: 0.020648 sec","job_status":"done","queue_duration_s":1.600098,"scheduling_latency_s":1.600123,"redis_calls":13,"redis_duration_s":0.002059,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000691,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001368,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013384,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.020648,"completed_at":"2026-10-02T16:59:32.424Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2256gitlab # [ 419.447518] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.425Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d75e95d82f5f58bdc35a87","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d75e95d82f5f58bdc35a87: start","job_status":"start","queue_duration_s":1.622028,"scheduling_latency_s":1.622046}2257gitlab # [ 419.460447] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.442Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"c2d75e95d82f5f58bdc35a87","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-c2d75e95d82f5f58bdc35a87: done: 0.017255 sec","job_status":"done","queue_duration_s":1.622028,"scheduling_latency_s":1.622046,"redis_calls":13,"redis_duration_s":0.002344,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000705,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001639,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011435,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.017255,"completed_at":"2026-10-02T16:59:32.442Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2258gitlab # [ 419.500680] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.443Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"838f581bfe6760aa129044fc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-838f581bfe6760aa129044fc: start","job_status":"start","queue_duration_s":1.640468,"scheduling_latency_s":1.640483}2259gitlab # [ 419.513420] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.460Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"838f581bfe6760aa129044fc","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-838f581bfe6760aa129044fc: done: 0.016334 sec","job_status":"done","queue_duration_s":1.640468,"scheduling_latency_s":1.640483,"redis_calls":13,"redis_duration_s":0.00198,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.00068,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.0013,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010822,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016334,"completed_at":"2026-10-02T16:59:32.460Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2260gitlab # [ 419.554475] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.461Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"194d8f7f45ab0d2d9aee5cb0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-194d8f7f45ab0d2d9aee5cb0: start","job_status":"start","queue_duration_s":1.6583,"scheduling_latency_s":1.658317}2261gitlab # [ 419.567369] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.477Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"194d8f7f45ab0d2d9aee5cb0","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-194d8f7f45ab0d2d9aee5cb0: done: 0.016053 sec","job_status":"done","queue_duration_s":1.6583,"scheduling_latency_s":1.658317,"redis_calls":13,"redis_duration_s":0.001989,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000663,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001326,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.010768,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016053,"completed_at":"2026-10-02T16:59:32.477Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2262gitlab # [ 419.607661] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.479Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ed3fd904ad3e9da31e8693","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ed3fd904ad3e9da31e8693: start","job_status":"start","queue_duration_s":1.675917,"scheduling_latency_s":1.675935}2263gitlab # [ 419.620588] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.495Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"b7ed3fd904ad3e9da31e8693","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-b7ed3fd904ad3e9da31e8693: done: 0.016294 sec","job_status":"done","queue_duration_s":1.675917,"scheduling_latency_s":1.675935,"redis_calls":13,"redis_duration_s":0.002024,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000713,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001311,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011331,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016294,"completed_at":"2026-10-02T16:59:32.495Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2264gitlab # [ 419.660810] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.497Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e50bda60acd75787e47285","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e50bda60acd75787e47285: start","job_status":"start","queue_duration_s":1.69378,"scheduling_latency_s":1.693798}2265gitlab # [ 419.673460] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.513Z","retry":0,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"Ci::UnlockPipelinesInQueueWorker","args":[],"jid":"62e50bda60acd75787e47285","created_at":"2026-10-02T16:59:30.707Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-ba3e49bca71c46f9","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"Ci::ScheduleUnlockPipelinesInQueueCronWorker","correlation_id":"161dfe8c49c99dd047007ae316c57f67","meta.root_caller_id":"Cronjob","meta.feature_category":"job_artifacts","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:30.803Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::UnlockPipelinesInQueueWorker JID-62e50bda60acd75787e47285: done: 0.016177 sec","job_status":"done","queue_duration_s":1.69378,"scheduling_latency_s":1.693798,"redis_calls":13,"redis_duration_s":0.00189,"redis_read_bytes":10,"redis_write_bytes":1157,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000694,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":359,"redis_shared_state_calls":9,"redis_shared_state_duration_s":0.001196,"redis_shared_state_read_bytes":7,"redis_shared_state_write_bytes":798,"db_count":0,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":0,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.0,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.011327,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"extra.ci_unlock_pipelines_in_queue_worker.remaining_pending":0,"duration_s":0.016177,"completed_at":"2026-10-02T16:59:32.513Z","load_balancing_strategy":"primary","db_duration_s":0.0,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2266gitlab # [ 419.713535] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.515Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a37be8326c3c2332374a720","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a37be8326c3c2332374a720: start","job_status":"start","queue_duration_s":1.31091,"scheduling_latency_s":1.310928}2267gitlab # [ 419.726105] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.534Z","message":"No existing merge request to be cleaned up."}2268gitlab # [ 419.727841] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.547Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"9a37be8326c3c2332374a720","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-9a37be8326c3c2332374a720: done: 0.032686 sec","job_status":"done","queue_duration_s":1.31091,"scheduling_latency_s":1.310928,"redis_calls":12,"redis_duration_s":0.002301,"redis_read_bytes":10,"redis_write_bytes":1323,"redis_feature_flag_calls":2,"redis_feature_flag_duration_s":0.000398,"redis_feature_flag_read_bytes":2,"redis_feature_flag_write_bytes":301,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000689,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001214,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":4,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":4,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.003,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":4,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.002,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.002,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.003,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.021935,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.032686,"completed_at":"2026-10-02T16:59:32.547Z","load_balancing_strategy":"primary","db_duration_s":0.004444,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2269gitlab # [ 419.770072] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.548Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5edafe8002dfdf65e1223869","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5edafe8002dfdf65e1223869: start","job_status":"start","queue_duration_s":1.344791,"scheduling_latency_s":1.344809}2270gitlab # [ 419.782816] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.559Z","message":"No existing merge request to be cleaned up."}2271gitlab # [ 419.784538] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.571Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"5edafe8002dfdf65e1223869","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-5edafe8002dfdf65e1223869: done: 0.022541 sec","job_status":"done","queue_duration_s":1.344791,"scheduling_latency_s":1.344809,"redis_calls":10,"redis_duration_s":0.002573,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.001618,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.000955,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.014344,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.022541,"completed_at":"2026-10-02T16:59:32.571Z","load_balancing_strategy":"primary","db_duration_s":0.002434,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2272gitlab # [ 419.824480] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.572Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e4b44868f480a5142f8e5850","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e4b44868f480a5142f8e5850: start","job_status":"start","queue_duration_s":1.368785,"scheduling_latency_s":1.368803}2273gitlab # [ 419.837076] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.587Z","message":"No existing merge request to be cleaned up."}2274gitlab # [ 419.838857] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.596Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"e4b44868f480a5142f8e5850","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-e4b44868f480a5142f8e5850: done: 0.023516 sec","job_status":"done","queue_duration_s":1.368785,"scheduling_latency_s":1.368803,"redis_calls":10,"redis_duration_s":0.002307,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000855,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.001452,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.002,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.001,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.001,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.01473,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.023516,"completed_at":"2026-10-02T16:59:32.596Z","load_balancing_strategy":"primary","db_duration_s":0.001974,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2275gitlab # [ 419.878581] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.597Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0cdc918fd53fc24597a065c3","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0cdc918fd53fc24597a065c3: start","job_status":"start","queue_duration_s":1.39361,"scheduling_latency_s":1.393628}2276gitlab # [ 419.891267] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.612Z","message":"No existing merge request to be cleaned up."}2277gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2278gitlab # [ 419.893077] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:32.622Z","retry":3,"queue":"default","version":0,"store":null,"status_expiration":1800,"class":"MergeRequestCleanupRefsWorker","args":[],"jid":"0cdc918fd53fc24597a065c3","created_at":"2026-10-02T16:59:31.191Z","trace_propagation_headers":{"sentry-trace":"9e5b3dd342304e188ada3985f733822f-6c638da4a3e04499","baggage":"sentry-trace_id=9e5b3dd342304e188ada3985f733822f,sentry-release=Unknown"},"meta.caller_id":"ScheduleMergeRequestCleanupRefsWorker","correlation_id":"f3a6aaf4c4af44deb13bd336064c4884","meta.root_caller_id":"Cronjob","meta.feature_category":"code_review_workflow","meta.client_id":"ip/","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"size_limiter":"validated","enqueued_at":"2026-10-02T16:59:31.204Z","job_size_bytes":2,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"MergeRequestCleanupRefsWorker JID-0cdc918fd53fc24597a065c3: done: 0.025111 sec","job_status":"done","queue_duration_s":1.39361,"scheduling_latency_s":1.393628,"redis_calls":10,"redis_duration_s":0.001818,"redis_read_bytes":8,"redis_write_bytes":1022,"redis_queues_metadata_calls":4,"redis_queues_metadata_duration_s":0.000768,"redis_queues_metadata_read_bytes":3,"redis_queues_metadata_write_bytes":353,"redis_shared_state_calls":6,"redis_shared_state_duration_s":0.00105,"redis_shared_state_read_bytes":5,"redis_shared_state_write_bytes":669,"db_count":3,"db_write_count":1,"db_cached_count":1,"db_txn_count":1,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":3,"db_replica_write_count":0,"db_primary_write_count":1,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":1,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":3,"db_ci_count":0,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":1,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.003,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.003,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.006,"db_ci_duration_s":0.0,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.013558,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.025111,"completed_at":"2026-10-02T16:59:32.622Z","load_balancing_strategy":"primary","db_duration_s":0.009356,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2279gitlab # [ 420.045350] gitaly[916]: time="2026-10-02T16:59:34.981Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX5442PP0JFJMF70DJSAJ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:04.579" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:34.980" grpc.time_ms=0.882 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2280gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.15 seconds)2281gitlab # [ 420.077775] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:35 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 962282gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2283gitlab # [ 421.179178] gitaly[916]: time="2026-10-02T16:59:36.115Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX67G7HDVHEGX43T4G7TY grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:06.013" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:36.113" grpc.time_ms=0.942 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2284gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.13 seconds)2285gitlab # [ 421.212642] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:36 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 639 "" "curl/8.21.0" 1002286gitlab # [ 421.222104] nix-daemon[3641]: accepted connection from pid 4731, user nixbot2287gitlab # [ 421.329334] gitaly[916]: time="2026-10-02T16:59:36.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:06.164" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:36.264" grpc.time_ms=0.882 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2288gitlab # [ 421.388409] gitaly[916]: time="2026-10-02T16:59:36.325Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=8 command.spawn_token_fork_ms=0 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:46.284" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.314" grpc.time_ms=10.364 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2289gitlab # [ 421.480376] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.415Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0ba847751b0fd234326e258d","created_at":"2026-10-02T16:59:36.406Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-3213078699d44926","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.412Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0ba847751b0fd234326e258d: start","job_status":"start","queue_duration_s":0.00294,"scheduling_latency_s":0.002985}2290gitlab # [ 421.506012] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 201 675 "" "python-httpx/0.28.1" 2452291gitlab # [ 421.510710] nixbot[3482]: {"ts": "2026-10-02T16:59:36.438Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 \"HTTP/1.1 201 Created\""}2292gitlab # [ 421.636471] gitaly[916]: time="2026-10-02T16:59:36.573Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6KVP80W43TH0F3D39TH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:06.171" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:36.571" grpc.time_ms=0.914 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2293gitlab # [ 421.658855] gitaly[916]: time="2026-10-02T16:59:36.595Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:36.595" grpc.time_ms=0.367 pid=916 span.kind=server system=grpc2294gitlab # [ 421.699527] gitaly[916]: time="2026-10-02T16:59:36.636Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3894 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:36.635" grpc.time_ms=0.572 pid=916 span.kind=server system=grpc2295gitlab # [ 421.737310] gitaly[916]: time="2026-10-02T16:59:36.673Z" level=info msg="finished streaming call with code OK" component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.client_name=gitaly-hooks grpc.meta.deadline_type=none grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=bidi_stream grpc.method=ReferenceTransactionHook grpc.request.fullMethod=/gitaly.HookService/ReferenceTransactionHook grpc.request.glProjectPath= grpc.request.glRepository= grpc.request.payload_bytes=3896 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=2 grpc.service=gitaly.HookService grpc.start_time="2026-10-02T16:59:36.673" grpc.time_ms=0.51 pid=916 span.kind=server system=grpc2296gitlab # [ 421.749208] gitaly[916]: time="2026-10-02T16:59:36.685Z" level=info msg="beginning scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW forced=false grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" operations="[repack_refs]" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2297gitlab # [ 421.761561] gitaly[916]: time="2026-10-02T16:59:36.687Z" level=info msg="repository info" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" pid=916 remote_ip="2001:db8:1::1" repository_info="{false {6 702 0 0 0 0} {0 0 0 0 0 0 0 0 0 {false 0 false false} {false 0 0 false false false} {false 0 false false} 2026-10-02 16:57:43.79972654 +0000 UTC} {3 0 [] 0 files} {false 0 false false false} {false [] 0001-01-01 00:00:00 +0000 UTC }}" span.kind=server system=grpc username=root2298gitlab # [ 421.774968] gitaly[916]: time="2026-10-02T16:59:36.688Z" level=info msg="diskcache state change" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW diskcache=5edcac01-f86a-48f1-ba7b-57baa8635bc0 grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2299gitlab # [ 421.785324] gitaly[916]: time="2026-10-02T16:59:36.689Z" level=info msg="finished unary call with code OK" command.count=1 command.cpu_time_ms=106 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=7221 command.oublock=0 command.real_time_ms=121 command.spawn_token_fork_ms=0 command.system_time_ms=46 command.user_time_ms=60 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=141 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" grpc.time_ms=127.201 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2300gitlab # [ 421.798990] gitaly[916]: time="2026-10-02T16:59:36.693Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=8 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=571 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=1 command.system_time_ms=8 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6KVP80W43TH0F3D39TH grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:46.171" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.681" grpc.time_ms=11.334 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2301gitlab # [ 421.813142] gitaly[916]: time="2026-10-02T16:59:36.712Z" level=info msg="removed files" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" pid=916 remote_ip="2001:db8:1::1" span.kind=server stale_data.configfullpath=0 stale_data.configkeys=0 stale_data.configsections=0 stale_data.infoattributes=0 stale_data.locks=0 stale_data.objectdirs=0 stale_data.objects=0 stale_data.packedrefslock=0 stale_data.packedrefsnew=0 stale_data.packfilelocks=0 stale_data.reflocks=0 stale_data.refs=0 stale_data.refsemptydir=0 stale_data.reftablelocks=0 stale_data.serverinfo=0 system=grpc username=root2302gitlab # [ 421.827012] gitaly[916]: time="2026-10-02T16:59:36.712Z" level=info msg="optimized repository" clean_stale_data_time=24.9898ms clean_work_trees_time="203.9µs" commit_graph_time=920ns correlation_id=01M3YRX6BZR9KMMRRYT8J225PW gl_project_path=root/test-flake gl_repo=project-1 optimizations="map[]" pack_refs_time="2.5µs" pid=916 prune_time=260ns relative_path=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git repack_time="1.66µs" storage_name=default system=housekeeping total_time=25.24998ms2303gitlab # [ 421.834224] gitaly[916]: time="2026-10-02T16:59:36.713Z" level=info msg="ended scheduled housekeeping" component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=mutator grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=DeleteRefs grpc.request.deadline="2026-10-02T17:00:06.561" grpc.request.fullMethod=/gitaly.RefService/DeleteRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:36.561" pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2304gitlab # [ 421.856422] gitaly[916]: time="2026-10-02T16:59:36.793Z" level=info msg="finished streaming call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListCommitsByOid grpc.request.deadline="2026-10-02T17:00:06.791" grpc.request.fullMethod=/gitaly.CommitService/ListCommitsByOid grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:36.791" grpc.time_ms=0.989 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2305gitlab # [ 421.906729] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:36 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 201 635 "" "python-httpx/0.28.1" 3992306gitlab # [ 421.911218] gitaly[916]: time="2026-10-02T16:59:36.844Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX6BZR9KMMRRYT8J225PW grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-sidekiq grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:06.842" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=129 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:36.843" grpc.time_ms=0.772 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2307gitlab # [ 421.931453] nixbot[3482]: {"ts": "2026-10-02T16:59:36.844Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 \"HTTP/1.1 201 Created\""}2308gitlab # [ 421.936211] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.868Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"8f0c9c5f71ffd5fb57f61eb7","created_at":"2026-10-02T16:59:36.859Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","duplicate-of":"0ba847751b0fd234326e258d","job_size_bytes":3,"pid":3364,"job_status":"deduplicated","message":"PipelineProcessWorker JID-8f0c9c5f71ffd5fb57f61eb7: deduplicated: until executed","deduplication.type":"until executed","deduplication.options.if_deduplicated":"reschedule_once","deduplication.options.ttl":60}2309gitlab # [ 421.958319] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.877Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0ba847751b0fd234326e258d","created_at":"2026-10-02T16:59:36.406Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-3213078699d44926","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.412Z","load_balancing_strategy":"primary","job_size_bytes":3,"pid":3364,"job_status":"rescheduled","message":"PipelineProcessWorker JID-0ba847751b0fd234326e258d: rescheduled"}2310gitlab # [ 421.976202] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.889Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_processing","args":["2"],"class":"PipelineProcessWorker","jid":"0ba847751b0fd234326e258d","created_at":"2026-10-02T16:59:36.406Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-3213078699d44926","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-environment=,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"POST /api/:version/projects/:id/statuses/:sha","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:eb86e5e3de0e02766c69db2e2b5051b955094568149296b3fa12a64076975b15","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.412Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineProcessWorker JID-0ba847751b0fd234326e258d: done: 0.473423 sec","job_status":"done","queue_duration_s":0.00294,"scheduling_latency_s":0.002985,"gitaly_calls":3,"gitaly_duration_s":0.140725,"redis_calls":79,"redis_allowed_cross_slot_calls":1,"redis_duration_s":0.033015,"redis_read_bytes":239,"redis_write_bytes":21784,"redis_action_cable_calls":4,"redis_action_cable_duration_s":0.001485,"redis_action_cable_read_bytes":4,"redis_action_cable_write_bytes":805,"redis_cache_calls":11,"redis_cache_allowed_cross_slot_calls":1,"redis_cache_duration_s":0.001602,"redis_cache_read_bytes":20,"redis_cache_write_bytes":948,"redis_queues_calls":25,"redis_queues_duration_s":0.005081,"redis_queues_read_bytes":28,"redis_queues_write_bytes":14469,"redis_queues_metadata_calls":23,"redis_queues_metadata_duration_s":0.019852,"redis_queues_metadata_read_bytes":112,"redis_queues_metadata_write_bytes":3754,"redis_rate_limiting_calls":1,"redis_rate_limiting_duration_s":0.000391,"redis_rate_limiting_read_bytes":1,"redis_rate_limiting_write_bytes":81,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000884,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":8,"redis_shared_state_duration_s":0.002235,"redis_shared_state_read_bytes":13,"redis_shared_state_write_bytes":785,"db_count":33,"db_write_count":3,"db_cached_count":1,"db_txn_count":2,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":33,"db_replica_write_count":0,"db_primary_write_count":3,"db_replica_cached_count":0,"db_primary_cached_count":1,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.078,"db_main_txn_count":0,"db_ci_txn_count":2,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":16,"db_ci_count":17,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":3,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":1,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.009,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.012,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.048,"db_ci_duration_s":0.03,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.235212,"worker_id":"sidekiq_0","rate_limiting_gates":["ci_pipeline_statuses_subscription"],"exclusive_lock_requested_count":2,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0.3779825,"duration_s":0.473423,"completed_at":"2026-10-02T16:59:36.888Z","load_balancing_strategy":"primary","db_duration_s":0.080421,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2311gitlab # [ 422.031328] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.892Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4f2147ba239db40edffab2f8","created_at":"2026-10-02T16:59:36.548Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.551Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4f2147ba239db40edffab2f8: start","job_status":"start","queue_duration_s":0.340619,"scheduling_latency_s":0.340651}2312gitlab # [ 422.048727] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.961Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_default","args":["2","[FILTERED]"],"class":"PipelineNotificationWorker","jid":"4f2147ba239db40edffab2f8","created_at":"2026-10-02T16:59:36.548Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:8bfbfb5f73a204e3fdd48256b83d50aefe68bf2f3ce2656861732de8a412dec3","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.551Z","job_size_bytes":23,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"PipelineNotificationWorker JID-4f2147ba239db40edffab2f8: done: 0.069345 sec","job_status":"done","queue_duration_s":0.340619,"scheduling_latency_s":0.340651,"redis_calls":7,"redis_duration_s":0.006084,"redis_read_bytes":5,"redis_write_bytes":661,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001896,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":461,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004188,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":200,"db_count":12,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":12,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.017,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":8,"db_ci_count":4,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.013,"db_ci_duration_s":0.003,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.044154,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.069345,"completed_at":"2026-10-02T16:59:36.961Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.016338,"urgency":"high","target_duration_s":10,"target_scheduling_latency_s":10}2313gitlab # [ 422.092893] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:36.963Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77041e6129951f57ffb69bc7","created_at":"2026-10-02T16:59:36.704Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.710Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77041e6129951f57ffb69bc7: start","job_status":"start","queue_duration_s":0.253679,"scheduling_latency_s":0.253709}2314gitlab # [ 422.109945] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.007Z","retry":25,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::JobArtifacts::TrackArtifactReportWorker","jid":"77041e6129951f57ffb69bc7","created_at":"2026-10-02T16:59:36.704Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"delayed","worker_data_consistency_per_db":{"main":"delayed","ci":"delayed"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:63e54343e7f7256ab8a1199953fd73b05666f28484b7429653936cc0ecd1a59c","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.710Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::JobArtifacts::TrackArtifactReportWorker JID-77041e6129951f57ffb69bc7: done: 0.04324 sec","job_status":"done","queue_duration_s":0.253679,"scheduling_latency_s":0.253709,"redis_calls":7,"redis_duration_s":0.005333,"redis_read_bytes":5,"redis_write_bytes":746,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.004812,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":512,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000521,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":234,"db_count":6,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":6,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.006,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.006,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.027659,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.04324,"completed_at":"2026-10-02T16:59:37.007Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.005893,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2315gitlab # [ 422.154665] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.009Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"473d7791e8df31d19f102dd2","created_at":"2026-10-02T16:59:36.719Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.725Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-473d7791e8df31d19f102dd2: start","job_status":"start","queue_duration_s":0.283993,"scheduling_latency_s":0.284018}2316gitlab # [ 422.171066] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.056Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CreateQualityReportWorker","jid":"473d7791e8df31d19f102dd2","created_at":"2026-10-02T16:59:36.719Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_quality","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:a438d79f5f5ba6d4ef2b849979aa33f2bcd0b8b093a7892d92863b57e28035e2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.725Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CreateQualityReportWorker JID-473d7791e8df31d19f102dd2: done: 0.046889 sec","job_status":"done","queue_duration_s":0.283993,"scheduling_latency_s":0.284018,"redis_calls":7,"redis_duration_s":0.002072,"redis_read_bytes":5,"redis_write_bytes":771,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001628,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":527,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000444,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":244,"db_count":8,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":8,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":0,"db_ci_count":8,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.0,"db_ci_duration_s":0.007,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.033701,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.046889,"completed_at":"2026-10-02T16:59:37.055Z","load_balancing_strategy":"primary","db_duration_s":0.007257,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2317gitlab: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses2318gitlab # [ 422.215225] gitaly[916]: time="2026-10-02T16:59:37.040Z" level=info msg="finished unary call with code OK" catfile.duration_ms=1 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=1 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX70M4GHVPGQSNJF24TRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:07.037" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:37.037" grpc.time_ms=2.358 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2319gitlab # [ 422.227807] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.058Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"358707db24f0ef81438057e7","created_at":"2026-10-02T16:59:36.737Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.742Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-358707db24f0ef81438057e7: start","job_status":"start","queue_duration_s":0.315666,"scheduling_latency_s":0.315695}2320gitlab # [ 422.244994] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.103Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::TestFailureHistoryWorker","jid":"358707db24f0ef81438057e7","created_at":"2026-10-02T16:59:36.737Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"continuous_integration","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"always","worker_data_consistency_per_db":{"main":"always","ci":"always"},"idempotency_key":"resque:gitlab:duplicate:default:721e749fe77618a4212007bc934d77ea6bfb5f3b79a2b9e7613972c1099ff3d2","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.742Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::TestFailureHistoryWorker JID-358707db24f0ef81438057e7: done: 0.045713 sec","job_status":"done","queue_duration_s":0.315666,"scheduling_latency_s":0.315695,"redis_calls":10,"redis_duration_s":0.0023290000000000003,"redis_read_bytes":62,"redis_write_bytes":808,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.001203,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":467,"redis_repository_cache_calls":3,"redis_repository_cache_duration_s":0.000661,"redis_repository_cache_read_bytes":57,"redis_repository_cache_write_bytes":137,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.000465,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":204,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.007,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":2,"db_ci_count":5,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.002,"db_ci_duration_s":0.005,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.032569,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.045713,"completed_at":"2026-10-02T16:59:37.103Z","load_balancing_strategy":"primary","db_duration_s":0.007334,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2321gitlab # [ 422.290360] gitaly[916]: time="2026-10-02T16:59:37.143Z" level=info msg="finished streaming call with code OK" command.count=2 command.cpu_time_ms=7 command.inblock=0 command.majflt=0 command.maxrssanon_bytes=0 command.minflt=572 command.oublock=0 command.real_time_ms=9 command.spawn_token_fork_ms=2 command.system_time_ms=7 command.user_time_ms=0 component=gitaly.StreamServerInterceptor correlation_id=01M3YRX70M4GHVPGQSNJF24TRS grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=server_stream grpc.method=ListRefs grpc.request.deadline="2026-10-02T16:59:47.101" grpc.request.fullMethod=/gitaly.RefService/ListRefs grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=135 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=0 grpc.service=gitaly.RefService grpc.start_time="2026-10-02T16:59:37.131" grpc.time_ms=12.147 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2322gitlab # [ 422.304326] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.105Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e9faaf04ab93e7b02087c056","created_at":"2026-10-02T16:59:36.743Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.746Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e9faaf04ab93e7b02087c056: start","job_status":"start","queue_duration_s":0.35882,"scheduling_latency_s":0.358844}2323gitlab # [ 422.321551] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.244Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"pipeline_background","args":["2"],"class":"Ci::PipelineArtifacts::CoverageReportWorker","jid":"e9faaf04ab93e7b02087c056","created_at":"2026-10-02T16:59:36.743Z","trace_propagation_headers":{"sentry-trace":"cda5d44326024e59802f2f4c569efd83-2226190ce3a94a35","baggage":"sentry-trace_id=cda5d44326024e59802f2f4c569efd83,sentry-release=Unknown"},"correlation_id":"01M3YRX6BZR9KMMRRYT8J225PW","meta.caller_id":"PipelineProcessWorker","meta.remote_ip":"2001:db8:1::1","meta.feature_category":"code_testing","meta.user":"root","meta.gl_user_id":1,"meta.project":"root/test-flake","meta.root_namespace":"root","meta.client_id":"user/1","meta.root_caller_id":"POST /api/:version/projects/:id/statuses/:sha","worker_data_consistency":"sticky","worker_data_consistency_per_db":{"main":"sticky","ci":"sticky"},"wal_locations":{},"wal_location_sources":{"main":"primary","ci":"primary"},"idempotency_key":"resque:gitlab:duplicate:default:99ba93e701690c3a10a017e4f432c18103a4ebef8dca1f80d289f0d8d281ddbd","size_limiter":"validated","enqueued_at":"2026-10-02T16:59:36.746Z","job_size_bytes":3,"pid":3364,"sidekiq_tid":"m8ng","sidekiq_thread_name":"sidekiq.default/processor","message":"Ci::PipelineArtifacts::CoverageReportWorker JID-e9faaf04ab93e7b02087c056: done: 0.139135 sec","job_status":"done","queue_duration_s":0.35882,"scheduling_latency_s":0.358844,"redis_calls":7,"redis_duration_s":0.016738,"redis_read_bytes":5,"redis_write_bytes":741,"redis_queues_metadata_calls":5,"redis_queues_metadata_duration_s":0.012189,"redis_queues_metadata_read_bytes":4,"redis_queues_metadata_write_bytes":509,"redis_shared_state_calls":2,"redis_shared_state_duration_s":0.004549,"redis_shared_state_read_bytes":1,"redis_shared_state_write_bytes":232,"db_count":7,"db_write_count":0,"db_cached_count":0,"db_txn_count":0,"db_replica_txn_count":0,"db_primary_txn_count":0,"db_replica_count":0,"db_primary_count":7,"db_replica_write_count":0,"db_primary_write_count":0,"db_replica_cached_count":0,"db_primary_cached_count":0,"db_replica_wal_count":0,"db_primary_wal_count":0,"db_replica_wal_cached_count":0,"db_primary_wal_cached_count":0,"db_replica_txn_max_duration_s":0.0,"db_primary_txn_max_duration_s":0.0,"db_replica_txn_duration_s":0.0,"db_primary_txn_duration_s":0.0,"db_replica_duration_s":0.0,"db_primary_duration_s":0.043,"db_main_txn_count":0,"db_ci_txn_count":0,"db_main_replica_txn_count":0,"db_ci_replica_txn_count":0,"db_main_count":1,"db_ci_count":6,"db_main_replica_count":0,"db_ci_replica_count":0,"db_main_write_count":0,"db_ci_write_count":0,"db_main_replica_write_count":0,"db_ci_replica_write_count":0,"db_main_cached_count":0,"db_ci_cached_count":0,"db_main_replica_cached_count":0,"db_ci_replica_cached_count":0,"db_main_wal_count":0,"db_ci_wal_count":0,"db_main_replica_wal_count":0,"db_ci_replica_wal_count":0,"db_main_wal_cached_count":0,"db_ci_wal_cached_count":0,"db_main_replica_wal_cached_count":0,"db_ci_replica_wal_cached_count":0,"db_main_txn_max_duration_s":0.0,"db_ci_txn_max_duration_s":0.0,"db_main_replica_txn_max_duration_s":0.0,"db_ci_replica_txn_max_duration_s":0.0,"db_main_txn_duration_s":0.0,"db_ci_txn_duration_s":0.0,"db_main_replica_txn_duration_s":0.0,"db_ci_replica_txn_duration_s":0.0,"db_main_duration_s":0.005,"db_ci_duration_s":0.038,"db_main_replica_duration_s":0.0,"db_ci_replica_duration_s":0.0,"cpu_s":0.031169,"worker_id":"sidekiq_0","rate_limiting_gates":[],"exclusive_lock_requested_count":1,"exclusive_lock_wait_duration_s":0,"exclusive_lock_hold_duration_s":0,"duration_s":0.139135,"completed_at":"2026-10-02T16:59:37.244Z","load_balancing_strategy":"primary_no_wal","db_duration_s":0.042862,"urgency":"low","target_duration_s":300,"target_scheduling_latency_s":60}2324gitlab: (finished: must succeed: TOKEN=$(cat /var/lib/secrets/gitlab-token); curl -fs -H "PRIVATE-TOKEN: $TOKEN" http://gitlab/api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses, in 0.15 seconds)2325(finished: subtest: eval and build post commit statuses, in 10.67 seconds)2326(finished: run the VM test script, in 490.24 seconds)2327gitlab # [ 422.366065] gitaly[916]: time="2026-10-02T16:59:37.265Z" level=info msg="finished unary call with code OK" catfile.duration_ms=0 catfile.flush_count=1 catfile.flush_ms=0 catfile.read_object_count=1 catfile.read_object_ms=0 catfile.request_object_count=1 catfile.request_object_ms=0 component=gitaly.UnaryServerInterceptor correlation_id=01M3YRX7BC2TAP5B4F736FPNMZ grpc.code=OK grpc.meta.auth_version=v2 grpc.meta.client_name=gitlab-web grpc.meta.deadline_type=regular grpc.meta.method_operation=accessor grpc.meta.method_scope=repository grpc.meta.method_type=unary grpc.method=FindCommit grpc.request.deadline="2026-10-02T17:00:07.264" grpc.request.fullMethod=/gitaly.CommitService/FindCommit grpc.request.glProjectPath=root/test-flake grpc.request.glRepository=project-1 grpc.request.payload_bytes=165 grpc.request.repoPath=@hashed/6b/86/6b86b273ff34fce19d6b804eff5a3f5747ada4eaa22f1d49c01e52ddb7875b4b.git grpc.request.repoStorage=default grpc.response.payload_bytes=244 grpc.service=gitaly.CommitService grpc.start_time="2026-10-02T16:59:37.265" grpc.time_ms=0.533 pid=916 remote_ip="2001:db8:1::1" span.kind=server system=grpc username=root2328gitlab # [ 422.379154] nixbot[3482]: {"ts": "2026-10-02T16:59:37.300Z", "level": "info", "logger": "httpx", "msg": "HTTP Request: POST http://gitlab/api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 \"HTTP/1.1 201 Created\""}2329gitlab # [ 422.383082] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:37 +0000] "GET /api/v4/projects/root%2Ftest-flake/repository/commits/4a967074777ce7606287d14f5463040267a12c45/statuses HTTP/1.1" 200 1334 "" "curl/8.21.0" 1002330gitlab # [ 422.388289] gitlab-workhorse[822]: gitlab 2001:db8:1::1 - - [2026/10/02:16:59:37 +0000] "POST /api/v4/projects/root%2Ftest-flake/statuses/4a967074777ce7606287d14f5463040267a12c45 HTTP/1.1" 201 656 "" "python-httpx/0.28.1" 4452331test script finished in 490.26s2332cleanup2333kill QemuMachine (pid 46)2334gitlab # [ 422.391656] sidekiq-cluster[3364]: {"severity":"INFO","time":"2026-10-02T16:59:37.306Z","retry":3,"queue":"default","version":0,"store":null,"queue_namespace":"auto_merge","args":["[FILTERED]"],"class":"AutoMergeProcessWorker","jid":"5366ecbd14057baed245c2f6","created_at":"2026-10-qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/41m77i1296n33p6liin8ynr6wh3h6b7m-python3-3.14.6/bin/python3.14)2335gitlab # 02336(finished: cleanup, in 1.64 seconds)2337additionally exposed symbols:2338 gitlab,2339 vlan1,2340 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh2341[{'id': 1, 'url': 'http://buildbot:8010/webhooks/gitlab', 'name': None, 'description': None, 'created_at': '2026-10-02T16:57:54.020Z', 'push_events': True, 'tag_push_events': False, 'merge_requests_events': True, 'repository_update_events': False, 'enable_ssl_verification': True, 'alert_status': 'executable', 'disabled_until': None, 'push_events_branch_filter': None, 'branch_filter_strategy': 'wildcard', 'custom_webhook_template': None, 'token_present': True, 'signing_token_present': False, 'project_id': 1, 'issues_events': False, 'confidential_issues_events': False, 'note_events': True, 'confidential_note_events': None, 'pipeline_events': False, 'wiki_page_events': False, 'deployment_events': False, 'feature_flag_events': False, 'job_events': False, 'releases_events': False, 'milestone_events': False, 'emoji_events': False, 'resource_access_token_events': False, 'resource_deploy_token_events': False, 'vulnerability_events': False}]2342[]2343[]2344[]2345[]2346[]2347[]2348[]2349[]2350[{'id': 2, 'project_id': 1, 'number': 2, 'tree_hash': 'cefc112e8680bb0f9569544b9656644ac3312d93', 'commit_sha': '4a967074777ce7606287d14f5463040267a12c45', 'branch': 'master', 'pr_number': None, 'pr_author': None, 'status': 'pending', 'eval_warnings': None, 'error': None, 'created_at': '2026-10-02T16:59:26.389423Z', 'started_at': None, 'finished_at': None, 'eval_duration_ms': None, 'eval_queue': {'position': 1, 'blocked_by': []}}]2351[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2352[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2353[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2354[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2355[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2356[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2357[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2358[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2359[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'pending', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'waiting for evaluation', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': None, 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]2360[{'id': 5, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-eval', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': 'evaluation succeeded in 9s (1 warning)', 'created_at': '2026-10-02T16:59:26.805Z', 'started_at': None, 'finished_at': '2026-10-02T16:59:36.387Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}, {'id': 6, 'sha': '4a967074777ce7606287d14f5463040267a12c45', 'ref': 'master', 'status': 'success', 'name': 'nixbot/nix-build', 'target_url': 'http://localhost:8010/repos/gitlab/root/test-flake/builds/2', 'description': '0 attributes built', 'created_at': '2026-10-02T16:59:36.785Z', 'started_at': None, 'finished_at': '2026-10-02T16:59:37.237Z', 'allow_failure': False, 'coverage': None, 'pipeline_id': 2, 'author': {'id': 1, 'username': 'root', 'public_email': None, 'name': 'Administrator', 'state': 'active', 'locked': False, 'avatar_url': 'https://www.gravatar.com/avatar/2e93f8045553c109d586f91d54e4902ff14dc793562a9d2ac2ca5bb86bce6ed9?s=80&d=identicon', 'web_url': 'http://gitlab/root'}}]